Changes for page SDMX 2.1 Standards. Section 6. Technical Notes
Last modified by Artur K. on 2026/05/29 14:28
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 0 added, 26 removed)
- 1747854006117-843.png
- 1747854039499-443.png
- 1747854067769-691.png
- 1747854096778-844.png
- 1747854127303-270.png
- 1747854163928-581.png
- 1747854190641-364.png
- 1747854236732-512.png
- 1747854286398-614.png
- 1747854385465-132.png
- 1747854406014-782.png
- 1747854424488-855.png
- 1747854453895-524.png
- 1747854476631-125.png
- 1747854493363-776.png
- 1747855024745-946.png
- 1747855054559-410.png
- 1747855075263-887.png
- 1747855402867-579.png
- 1747855462293-368.png
- 1747855493531-357.png
- 1747856459132-767.png
- 1747859398930-848.png
- 1747859458410-183.png
- 1747859612718-454.png
- 1747859722732-549.png
Details
- Page properties
-
- Content
-
... ... @@ -4,106 +4,123 @@ 4 4 5 5 **Revision History** 6 6 7 -(% style="width:954.835px" %) 8 -|(% style="width:106px" %)**Revision**|(% style="width:124px" %)**Date**|(% style="width:723px" %)**Contents** 9 -|(% style="width:106px" %) |(% style="width:124px" %)April 2011|(% style="width:723px" %)Initial release 10 -|(% style="width:106px" %)1.0|(% style="width:124px" %)April 2013|(% style="width:723px" %)Added section 9 - Transforming between versions of SDMX 11 -|(% style="width:106px" %)2.0|(% style="width:124px" %)July 2020|(% style="width:723px" %)Added section 10 – Validation and Transformation Language – before the Annex 1. 7 +|**Revision**|**Date**|**Contents** 8 +| |April 2011|Initial release 9 +|1.0|April 2013|Added section 9 - Transforming between versions of SDMX 10 +|2.0|July 2020|Added section 10 – Validation and Transformation Language – before the Annex 1. 12 12 13 13 = 1 Purpose and Structure = 14 14 15 15 == 1.1 Purpose == 16 16 17 -The intention of this document is to document certain aspects of [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]thatare important to understand and will aid implementation decisions. The explanations here supplement the information documented in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]XML schema and theInformation Model.16 +The intention of this document is to document certain aspects of SDMX that are important to understand and will aid implementation decisions. The explanations here supplement the information documented in the SDMX XML schema and the 18 18 18 +Information Model. 19 + 19 19 == 1.2 Structure == 20 20 21 21 This document is organized into the following major parts: 22 22 23 -A guide to the [[SDMX Information Model>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]] relating to[[Data Structure Definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and[[Data Sets>>doc:sdmx:Glossary.Data set.WebHome]], statement of differences in functionality supported by the different formats and syntaxes for[[Data Structure Definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and[[Data Sets>>doc:sdmx:Glossary.Data set.WebHome]], and best practices for use of[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]formats, including the[[representation>>doc:sdmx:Glossary.Representation.WebHome]]for time period24 +A guide to the SDMX Information Model relating to Data Structure Definitions and Data Sets, statement of differences in functionality supported by the different formats and syntaxes for Data Structure Definitions and Data Sets, and best practices for use of SDMX formats, including the representation for time period 24 24 25 -A guide to the [[SDMX Information Model>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]] relating to[[Metadata Structure Definitions>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], and Metadata Sets26 +A guide to the SDMX Information Model relating to Metadata Structure Definitions, and Metadata Sets 26 26 27 -Other structural [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]of interest: agencies,(% style="color:#e74c3c" %)concept(%%)role.(% style="color:#e74c3c" %)constraint(%%), partial[[code list>>doc:sdmx:Glossary.Code list.WebHome]]28 +Other structural artefacts of interest: agencies, concept role. constraint, partial code list 28 28 29 29 = 2 General Notes on This Document = 30 30 31 -At this (% style="color:#e74c3c" %)version(%%)of the standards, the term “Key family” is replaced by[[Data Structure Definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]](also known and referred to as[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]) both in the XML schemas and the Information Model. The term “Key family” is not familiar to many people and its name was taken from the model of[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]](previously known as GESMES/TS). The more familiar name “[[Data Structure Definition>>doc:sdmx:Glossary.Data structure definition.WebHome]]” which was used in many documents is now also the technical[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]in the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]and Information Model technical specifications. The term “Key family” is still used in the[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]specification.32 +At this version of the standards, the term “Key family” is replaced by Data Structure Definition (also known and referred to as DSD) both in the XML schemas and the Information Model. The term “Key family” is not familiar to many people and its name was taken from the model of SDMX-EDI (previously known as GESMES/TS). The more familiar name “Data Structure Definition” which was used in many documents is now also the technical artefact in the SDMX-ML and Information Model technical specifications. The term “Key family” is still used in the SDMX-EDI specification. 32 32 33 -There has been much work within the [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] community on the creation of user guides, tutorials, and other aides to implementation and understanding of the standard. This document is not intended to duplicate the function of these documents, but instead represents a short set of technical notes not generally covered elsewhere.34 +There has been much work within the SDMX community on the creation of user guides, tutorials, and other aides to implementation and understanding of the standard. This document is not intended to duplicate the function of these documents, but instead represents a short set of technical notes not generally covered elsewhere. 34 34 35 35 = 3 Guide for SDMX Format Standards = 36 36 37 37 == 3.1 Introduction == 38 38 39 -This guide exists to provide information to implementers of the [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]format standards –[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]and[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]– that are concerned with data, i.e.[[Data Structure Definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and[[Data Sets>>doc:sdmx:Glossary.Dataset.WebHome]].This section is intended to provide information which will help users of[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]understand and implement the standards. It is not normative, and it does not provide any rules for the use of the standards, such as those found in //[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]:Schema and Documentation// and //[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]:Syntax and Documentation//.40 +This guide exists to provide information to implementers of the SDMX format standards – SDMX-ML and SDMX-EDI – that are concerned with data, i.e. Data Structure Definitions and Data Sets. This section is intended to provide information which will help users of SDMX understand and implement the standards. It is not normative, and it does not provide any rules for the use of the standards, such as those found in //SDMX-ML: Schema and Documentation// and //SDMX-EDI: Syntax and Documentation//. 40 40 41 41 == 3.2 SDMX Information Model for Format Implementers == 42 42 43 -=== 3.2.1 Introduction === 44 +=== 3.2.1 Introduction === 44 44 45 -The purpose of this sub-section is to provide an introduction to the [[SDMX-IM>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]] relating to[[Data Structure Definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and[[Data Sets>>doc:sdmx:Glossary.Dataset.WebHome]]for those whose primary interest is in the use of the XML or EDI formats. For those wishing to have a deeper understanding of the Information Model, the full[[SDMX-IM>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]] document, and other sections in this guide provide a more in-depth view, along with UML diagrams and supporting explanation. For those who are unfamiliar with DSDs, an appendix to the[[SDMX-IM>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]]provides a tutorial which may serve as a useful introduction.46 +The purpose of this sub-section is to provide an introduction to the SDMX-IM relating to Data Structure Definitions and Data Sets for those whose primary interest is in the use of the XML or EDI formats. For those wishing to have a deeper understanding of the Information Model, the full SDMX-IM document, and other sections in this guide provide a more in-depth view, along with UML diagrams and supporting explanation. For those who are unfamiliar with DSDs, an appendix to the SDMX-IM provides a tutorial which may serve as a useful introduction. 46 46 47 -The [[SDMX-IM>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]] is used to describe the basic data and metadata structures used in all of the[[SDMX>>doc:sdmx:Glossary.Statisticaldataand metadata exchange.WebHome]] dataformats. The Information Model concerns itself with statistical data and its[[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]], and that is what is described here. Both[[structural metadata>>doc:sdmx:Glossary.Structuralmetadata.WebHome]] and data have some additional metadata in common, related to their management and administration. These aspects of the data model are not addressed in this section and covered elsewhere in this guide or in the full[[SDMX-IM>>doc:sdmx:Glossary.SDMXInformation Model.WebHome]] document.48 +The SDMX-IM is used to describe the basic data and metadata structures used in all of the SDMX data formats. The Information Model concerns itself with statistical data and its structural metadata, and that is what is described here. Both structural metadata and data have some additional metadata in common, related to their management and administration. These aspects of the data model are not addressed in this section and covered elsewhere in this guide or in the full SDMX-IM document. 48 48 49 -The [[Data Structure Definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and[[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]parts of the information model are consistent with the GESMES/TS(% style="color:#e74c3c" %)version(%%)3.0 Data Model (called[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]standard), with these exceptions:50 +The Data Structure Definition and Data Set parts of the information model are consistent with the GESMES/TS version 3.0 Data Model (called SDMX-EDI in the SDMX standard), with these exceptions: 50 50 51 -* the “[[sibling group>>doc:sdmx:Glossary.Sibling group.WebHome]]” construct has been generalized to permit any [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] or [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] to be wildcarded, and not just frequency, as in GESMES/TS. It has been renamed a “group” to distinguish it from the “[[sibling group>>doc:sdmx:Glossary.Sibling group.WebHome]]” where only frequency is wildcarded. The set of allowable partial “group” keys must be declared in the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], and [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] may be attached to any of these [[group keys>>doc:sdmx:Glossary.Group key.WebHome]]; 52 -* furthermore, whilst the “group” has been retained for compatibility with (% style="color:#e74c3c" %)version(%%) 2.0 and with [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]], it has, at (% style="color:#e74c3c" %)version(%%) 2.1, been replaced by the “[[Attribute Relationship>>doc:sdmx:Glossary.Attribute relationship.WebHome]]” definition which is explained later 53 -* the section on data [[representation>>doc:sdmx:Glossary.Representation.WebHome]] is now a convention, to support interoperability with EDIFACT-syntax implementations ( see section 3.3.2); 52 +the “sibling group” construct has been generalized to permit any dimension or dimensions to be wildcarded, and not just frequency, as in GESMES/TS. It has been renamed a “group” to distinguish it from the “sibling group” where only frequency is wildcarded. The set of allowable partial “group” keys must be declared in the DSD, and attributes may be attached to any of these group keys; 54 54 55 - [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]-specific data formats arederived from the model, andsome supportingfeaturesfor declaringmultiple [[measures>>doc:sdmx:Glossary.Measure.WebHome]]havebeen addedtothe [[structuralmetadata>>doc:sdmx:Glossary.Structuralmetadata.WebHome]] descriptions Clearly,thisisnotacoincidence.TheGESMES/TSDataModel providesthefoundationfortheEDIFACT messages in[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]],andalsoisthestarting pointfor thedevelopmentof[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]].54 +furthermore, whilst the “group” has been retained for compatibility with version 2.0 and with SDMX-EDI, it has, at version 2.1, been replaced by the “Attribute Relationship” definition which is explained later 56 56 57 - Note thatinthedescriptionsbelow,textincourieranditalicisedarethenames used inthe informationmodel(e.g.//[[DataSet>>doc:sdmx:Glossary.Dataset.WebHome]]//).56 +the section on data representation is now a convention, to support interoperability with EDIFACT-syntax implementations ( see section 3.3.2); 58 58 58 +DSD-specific data formats are derived from the model, and some supporting features for declaring multiple measures have been added to the structural metadata descriptions 59 + 60 +Clearly, this is not a coincidence. The GESMES/TS Data Model provides the foundation for the EDIFACT messages in SDMX-EDI, and also is the starting point for the development of SDMX-ML. 61 + 62 +Note that in the descriptions below, text in courier and italicised are the names used in the information model (e.g. //DataSet//). 63 + 59 59 == 3.3 SDMX-ML and SDMX-EDI: Comparison of Expressive Capabilities and Function == 60 60 61 - [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] offers several equivalent formats for describing data and[[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]], optimized for use in different applications. Although all of these formats are derived directly from the SDM-IM, and are thus equivalent, the syntaxes used to express the model place some restrictions on their use. Also, different optimizations provide different capabilities. This section describes these differences, and provides some rules for applications which may need to support more than one[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]format or syntax. This section is constrained to the Data Structure Definitionand the Date Set.66 +SDMX offers several equivalent formats for describing data and structural metadata, optimized for use in different applications. Although all of these formats are derived directly from the SDM-IM, and are thus equivalent, the syntaxes used to express the model place some restrictions on their use. Also, different optimizations provide different capabilities. This section describes these differences, and provides some rules for applications which may need to support more than one SDMX format or syntax. This section is constrained to the Data Structure Definitionand the Date Set. 62 62 63 -=== 3.3.1 Format Optimizations and Differences === 68 +=== 3.3.1 Format Optimizations and Differences === 64 64 65 -The following section provides a brief overview of the differences between the various [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]formats.70 +The following section provides a brief overview of the differences between the various SDMX formats. 66 66 67 - (% style="color:#e74c3c" %)Version(%%)2.0 was characterised by 4 data messages, each with a distinct format: Generic, Compact, Cross-Sectional and Utility. Because of the design, data in some formats could not always be related to another format. In(% style="color:#e74c3c" %)version(%%)2.1, this issue has been addressed by merging some formats and eliminating others. As a result, in[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] 2.1 there are just two types of data formats: //GenericData// and //StructureSpecificData// (i.e. specific to one [[Data Structure Definition>>doc:sdmx:Glossary.Data structure definition.WebHome]]).72 +Version 2.0 was characterised by 4 data messages, each with a distinct format: Generic, Compact, Cross-Sectional and Utility. Because of the design, data in some formats could not always be related to another format. In version 2.1, this issue has been addressed by merging some formats and eliminating others. As a result, in 68 68 69 - Bothoftheseformats arenow flexible enoughtoallowfordata to be oriented in series with any[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]used to disambiguate the observations(asopposed to only time ora cross sectional [[measure>>doc:sdmx:Glossary.Measure.WebHome]] in(% style="color:#e74c3c" %)version(%%) 2.0). The formats havealso beenexpanded to allow for ungrouped observations.74 +SDMX 2.1 there are just two types of data formats: //GenericData// and 70 70 71 - To allow for applications which only understand time series data, variations of these formats have been introduced in the form of two data messages;//GenericTimeSeriesData// and //StructureSpecificTimeSeriesData//.Itis important to notethat thesevariationsare built onthesame rootstructureand can beprocessed in the same manner as the baseformat so that they do NOTintroduce additional processing requirements.76 +//StructureSpecificData// (i.e. specific to one Data Structure Definition). 72 72 73 - **//StructureDefinition//**78 +Both of these formats are now flexible enough to allow for data to be oriented in series with any dimension used to disambiguate the observations (as opposed to only time or a cross sectional measure in version 2.0). The formats have also been expanded to allow for ungrouped observations. 74 74 75 -T he [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]StructureMessage supportstheuseof[[annotations>>doc:sdmx:Glossary.Annotation.WebHome]]to thestructure,whichisnotsupportedbythe[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]syntax.80 +To allow for applications which only understand time series data, variations of these formats have been introduced in the form of two data messages; 76 76 77 - The [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] StructureMessage allows forthestructures on which a [[DataStructure Definition>>doc:sdmx:Glossary.Datastructuredefinition.WebHome]] depends–that is,codelistsand(% style="color:#e74c3c"%)concepts(%%) – tobeeitherincluded inthemessageortobereferencedbythemessage containingthe[[datastructuredefinition>>doc:sdmx:Glossary.Datastructure definition.WebHome]].XMLsyntax isdesignedto leverageURIsand otherInternet-basedreferencingmechanisms,andtheseare used inthe [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]message. This optionis notavailabletothoseusingthe [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]] structuremessage.82 +//GenericTimeSeriesData// and //StructureSpecificTimeSeriesData//. It is important to note that these variations are built on the same root structure and can be processed in the same manner as the base format so that they do NOT introduce additional processing requirements. 78 78 79 - **//Validation//**84 +=== //Structure Definition// === 80 80 81 - [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]– as istypical of EDIFACT syntaxmessages– leavesvalidation to dedicated applications(“validation” being thecheckingofsyntax, data typing, and adherenceofthe data messageto the structureasdescribedinthestructuraldefinition.)86 +The SDMX-ML Structure Message supports the use of annotations to the structure, which is not supported by the SDMX-EDI syntax. 82 82 83 -The [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]GenericDataMessagealsoleavesvalidationabove the XML syntax(%style="color:#e74c3c"%)level(%%)tothe application.88 +The SDMX-ML Structure Message allows for the structures on which a Data Structure Definition depends – that is, codelists and concepts – to be either included in the message or to be referenced by the message containing the data structure definition. XML syntax is designed to leverage URIs and other Internet-based referencing mechanisms, and these are used in the SDMX-ML message. This option is not available to those using the SDMX-EDI structure message. 84 84 85 - The[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]-specific messages will allow validationof XML syntax and datatyping to be performed with a generic XML parser, and enforce agreement between the structural definition and the data to a moderate degree with the same tool.90 +=== //Validation// === 86 86 87 - //UpdateandDeleteMessages andDocumentationMessages//92 +SDMX-EDI – as is typical of EDIFACT syntax messages – leaves validation to dedicated applications (“validation” being the checking of syntax, data typing, and adherence of the data message to the structure as described in the structural 88 88 89 - All [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] data messages allowfor both delete messages and messages consistingof only data or only documentation.94 +definition.) 90 90 91 - **//CharacterEncodings//**96 +The SDMX-ML Generic Data Message also leaves validation above the XML syntax level to the application. 92 92 93 - All[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]messagesuse the UTF-8 encoding,while[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]uses the ISO 8879-1 character encoding.Thereis agreater capacitywithUTF-8to express somecharacter sets (see the“APPENDIX:(% style="color:#e74c3c" %)MAP(%%) OF ISO 8859-1 (UNOC) CHARACTER SET (LATIN 1 OR “WESTERN”) inthedocument“SYNTAXAND DOCUMENTATION(%style="color:#e74c3c"%)VERSION(%%) 2.0”.) ManytransformationtoolsareavailablewhichallowXMLinstanceswithUTF-8encodingstobe expressedasISO 8879-1-encoded characters,andto transformUTF-8into ISO 8879-1. Such tools should beused when transforming [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]messages into[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]] messages and vice-versa.98 +The SDMX-ML DSD-specific messages will allow validation of XML syntax and datatyping to be performed with a generic XML parser, and enforce agreement between the structural definition and the data to a moderate degree with the same tool. 94 94 95 - **//DataTyping//**100 +=== //Update and Delete Messages and Documentation Messages// === 96 96 97 - TheXMLsyntax andEDIFACT syntax have differentdata-typingmechanisms. Thesectionbelowprovides a setof conventionsto beobservedwhensupport formessagesin both syntaxesisrequired.For more informationon the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]][[representations>>doc:sdmx:Glossary.Representation.WebHome]] of data, see below.102 +All SDMX data messages allow for both delete messages and messages consisting of only data or only documentation. 98 98 99 -=== 3.3.2 DataTypes ===104 +=== //Character Encodings// === 100 100 101 - TheXML syntax hasaverydifferentmechanism fordata-typingthan the EDIFACTsyntax,andthis differencemaycreate some difficulties forapplications which support both EDIFACT-basedandXML-based [[SDMX>>doc:sdmx:Glossary.Statisticaldataand metadataexchange.WebHome]] data formats. This sectionprovides a setofconventionsfor theexpressionin data in all formats,toallow forclean interoperabilitybetweenthem.106 +All SDMX-ML messages use the UTF-8 encoding, while SDMX-EDI uses the ISO 8879-1 character encoding. There is a greater capacity with UTF-8 to express some character sets (see the “APPENDIX: MAP OF ISO 8859-1 (UNOC) CHARACTER 102 102 103 - Itshouldbenotedthatthissectiondoesnotaddresscharacterencodings–itisassumedthatconversionsoftwarewillincludetheuseoftransformationswhichwill(%style="color:#e74c3c"%)map(%%)betweentheISO8879-1encodingofthe [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]] format andtheUTF-8encodingofthe[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]formats.108 +SET (LATIN 1 OR “WESTERN”) in the document “SYNTAX AND 104 104 105 -N otethatthefollowing conventionsmay befollowedforeaseofinteroperationbetweenEDIFACTandXML[[representations>>doc:sdmx:Glossary.Representation.WebHome]]of the data andmetadata. For implementationsinwhichnotransformationbetween EDIFACT andXML syntaxes is foreseen,therestrictionsbelowneednotapply.110 +DOCUMENTATION VERSION 2.0”.) Many transformation tools are available which allow XML instances with UTF-8 encodings to be expressed as ISO 8879-1-encoded characters, and to transform UTF-8 into ISO 8879-1. Such tools should be used when transforming SDMX-ML messages into SDMX-EDI messages and vice-versa. 106 106 112 +=== //Data Typing// === 113 + 114 +The XML syntax and EDIFACT syntax have different data-typing mechanisms. The section below provides a set of conventions to be observed when support for messages in both syntaxes is required. For more information on the SDMX-ML representations of data, see below. 115 + 116 +==== 3.3.2 Data Types ==== 117 + 118 +The XML syntax has a very different mechanism for data-typing than the EDIFACT syntax, and this difference may create some difficulties for applications which support both EDIFACT-based and XML-based SDMX data formats. This section provides a set of conventions for the expression in data in all formats, to allow for clean interoperability between them. 119 + 120 +It should be noted that this section does not address character encodings – it is assumed that conversion software will include the use of transformations which will map between the ISO 8879-1 encoding of the SDMX-EDI format and the UTF-8 encoding of the SDMX-ML formats. 121 + 122 +Note that the following conventions may be followed for ease of interoperation between EDIFACT and XML representations of the data and metadata. For implementations in which no transformation between EDIFACT and XML syntaxes is foreseen, the restrictions below need not apply. 123 + 107 107 1. **Identifiers** are: 108 108 1*. Maximum 18 characters; 109 109 1*. Any of A..Z (upper case alphabetic), 0..9 (numeric), _ (underscore); ... ... @@ -112,220 +112,220 @@ 112 112 1*. Maximum 70 characters. 113 113 1*. From ISO 8859-1 character set (including accented characters) 114 114 1. **Descriptions **are: 115 -1*. Maximum 350 characters; 116 -1*. From ISO 8859-1 character set. 132 +1*. Maximum 350 characters; From ISO 8859-1 character set. 117 117 1. **Code values** are: 118 118 1*. Maximum 18 characters; 119 119 1*. Any of A..Z (upper case alphabetic), 0..9 (numeric), _ (underscore), / (solidus, slash), = (equal sign), - (hyphen); 120 120 121 -However, [[code>>doc:sdmx:Glossary.Code.WebHome]]values providing values to a[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]must use only the following characters:137 +However, code values providing values to a dimension must use only the following characters: 122 122 123 123 A..Z (upper case alphabetic), 0..9 (numeric), _ (underscore) 124 124 125 -**5. Observation values** are: 141 +1. **Observation values** are: 142 +1*. Decimal numerics (signed only if they are negative); 143 +1*. The maximum number of significant figures is: 144 +1*. 15 for a positive number 145 +1*. 14 for a positive decimal or a negative integer 146 +1*. 13 for a negative decimal 147 +1*. Scientific notation may be used. 148 +1. **Uncoded statistical concept** text values are: 149 +1*. 150 +1**. Maximum 1050 characters; 151 +1**. From ISO 8859-1 character set. 152 +1. **Time series keys**: 126 126 127 -* [[Decimal>>doc:sdmx:Glossary.Decimals.WebHome]] numerics (signed only if they are negative); 128 -* The maximum number of significant figures is: 129 -* 15 for a positive number 130 -* 14 for a positive [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]] or a negative integer 131 -* 13 for a negative [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]] 132 -* Scientific notation may be used. 154 +In principle, the maximum permissible length of time series keys used in a data exchange does not need to be restricted. However, for working purposes, an effort is made to limit the maximum length to 35 characters; in this length, also (for SDMXEDI) one (separator) position is included between all successive dimension values; this means that the maximum length allowed for a pure series key (concatenation of dimension values) can be less than 35 characters. The separator character is a colon (“:”) by conventional usage. 133 133 134 -**6. Uncoded statistical concept** text values are: 135 - 136 -* Maximum 1050 characters; 137 -* From ISO 8859-1 character set. 138 - 139 -**7. Time series keys**: 140 - 141 -In principle, the maximum permissible length of time [[series keys>>doc:sdmx:Glossary.Series key.WebHome]] used in a data exchange does not need to be restricted. However, for working purposes, an effort is made to limit the maximum length to 35 characters; in this length, also (for SDMXEDI) one (separator) position is included between all successive [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] values; this means that the maximum length allowed for a pure [[series key>>doc:sdmx:Glossary.Series key.WebHome]] (concatenation of [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] values) can be less than 35 characters. The separator character is a colon (“:”) by conventional usage. 142 - 143 143 == 3.4 SDMX-ML and SDMX-EDI Best Practices == 144 144 145 -=== 3.4.1 Reporting and Dissemination Guidelines === 158 +=== 3.4.1 Reporting and Dissemination Guidelines === 146 146 147 - ====3.4.1.1 Central Institutions and Their Role in Statistical Data Exchanges====160 +**3.4.1.1 Central Institutions and Their Role in Statistical Data Exchanges **Central institutions are the organisations to which other partner institutions "report" statistics. These statistics are used by central institutions either to compile aggregates and/or they are put together and made available in a uniform manner (e.g. on-line or on a CD-ROM or through file transfers). Therefore, central institutions receive data from other institutions and, usually, they also "disseminate" data to individual and/or institutions for end-use. Within a country, a NSI or a national central bank (NCB) plays, of course, a central institution role as it collects data from other entities and it disseminates statistical information to end users. In SDMX the role of central institution is very important: every statistical message is based on underlying structural definitions (statistical concepts, code lists, DSDs) which have been devised by a particular agency, usually a central institution. Such an institution plays the role of the reference "structural definitions maintenance agency" for the corresponding messages which are exchanged. Of course, two institutions could exchange data using/referring to structural information devised by a third institution. 148 148 149 -Central institutions are the organisations to which other partner institutions "report" statistics. These statistics are used by central institutions either to compile aggregates and/or they are put together and made available in a uniform manner (e.g. on-line or on a CD-ROM or through file transfers). Therefore, central institutions receive data from other institutions and, usually, they also "disseminate" data to individual and/or institutions for end-use. Within a country, a NSI or a national central bank (NCB) plays, of course, a central institution role as it collects data from other entities and it disseminates statistical information to end users. In [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] the role of central institution is very important: every statistical message is based on underlying structural definitions (statistical (% style="color:#e74c3c" %)concepts(%%), [[code lists>>doc:sdmx:Glossary.Code list.WebHome]], DSDs) which have been devised by a particular agency, usually a central institution. Such an institution plays the role of the reference "structural definitions [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]] for the corresponding messages which are exchanged. Of course, two institutions could exchange data using/referring to structural information devised by a third institution. 150 - 151 151 Central institutions can play a double role: 152 152 153 153 * collecting and further disseminating statistics; 154 154 * devising structural definitions for use in data exchanges. 155 155 156 - ====3.4.1.2 Defining Data Structure Definitions (DSDs)====167 +**3.4.1.2 Defining Data Structure Definitions (DSDs)** 157 157 158 -The following guidelines are suggested for building a [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]. However, it is expected that these guidelines will be considered by central institutions when devising new DSDs.169 +The following guidelines are suggested for building a DSD. However, it is expected that these guidelines will be considered by central institutions when devising new DSDs. 159 159 160 -(% class="wikigeneratedid" id="HDimensions2CAttributesandCodeLists" %) 161 -__Dimensions, Attributes and Code Lists__ 171 +=== Dimensions, Attributes and Code Lists === 162 162 163 -**//Avoid [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]that are not appropriate for all the series in the[[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]].//** If some[[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]are not applicable (this is evident from the need to have a[[code>>doc:sdmx:Glossary.Code.WebHome]]in a[[code list>>doc:sdmx:Glossary.Codelist.WebHome]]which is marked as “not applicable”, “not relevant” or “total”) for some series then consider moving these series to a new[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]]inwhich these[[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]are dropped from the key structure. This is a judgement call as it is sometimes difficult to achieve this without increasing considerably the number of DSDs.173 +**//Avoid dimensions that are not appropriate for all the series in the data structure definition.//** If some dimensions are not applicable (this is evident from the need to have a code in a code list which is marked as “not applicable”, “not relevant” or “total”) for some series then consider moving these series to a new data structure definition in which these dimensions are dropped from the key structure. This is a judgement call as it is sometimes difficult to achieve this without increasing considerably the number of DSDs. 164 164 165 -**//Devise DSDs with a small number of [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]for public viewing of data.//** A[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]with the number[[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]in excess 6 or 7 is often difficult for non specialist users to understand. In these cases it is better to have a larger number of DSDs with smaller “cubes” of data, or to eliminate[[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]and aggregate the data at a higher(% style="color:#e74c3c" %)level(%%). Dissemination of data on the web is a growing use case for the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]standards: the differentiation of observations by dimensionality which are necessary for statisticians and economists are often obscure to public consumers who may not always understand the semantic of the differentiation.175 +**//Devise DSDs with a small number of Dimensions for public viewing of data.//** A DSD with the number dimensions in excess 6 or 7 is often difficult for non specialist users to understand. In these cases it is better to have a larger number of DSDs with smaller “cubes” of data, or to eliminate dimensions and aggregate the data at a higher level. Dissemination of data on the web is a growing use case for the SDMX standards: the differentiation of observations by dimensionality which are necessary for statisticians and economists are often obscure to public consumers who may not always understand the semantic of the differentiation. 166 166 167 -**//Avoid composite [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]].//** Each[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]should correspond to a single characteristic of the data, not to a combination of characteristics.177 +**//Avoid composite dimensions.//** Each dimension should correspond to a single characteristic of the data, not to a combination of characteristics. 168 168 169 -**//Consider the inclusion of the following [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]//**. Once the key structure of a[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]]has been decided, then the set of (preferably mandatory)[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]of this[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]]has to be defined. In general, some statistical(% style="color:#e74c3c" %)concepts(%%)are deemed necessary across all[[Data Structure Definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]toqualify the contained information. Examples of these are:179 +**//Consider the inclusion of the following attributes//**. Once the key structure of a data structure definition has been decided, then the set of (preferably mandatory) attributes of this data structure definition has to be defined. In general, some statistical concepts are deemed necessary across all Data Structure Definitions to qualify the contained information. Examples of these are: 170 170 171 -* A descriptive [[title>>doc:sdmx:Glossary.Title.WebHome]]for the series (this is most useful for dissemination of data for viewing e.g. on the web)181 +* A descriptive title for the series (this is most useful for dissemination of data for viewing e.g. on the web) 172 172 * Collection (e.g. end of period, averaged or summed over period) 173 -* Unit (e.g. [[currency>>doc:sdmx:Glossary.Currency.WebHome]]of denomination)183 +* Unit (e.g. currency of denomination) 174 174 * Unit multiplier (e.g. expressed in millions) 175 175 * Availability (which institutions can a series become available to) 176 -* Decimals (i.e. number of [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]]digits used in numerical observations)177 -* [[Observation Status>>doc:sdmx:Glossary.Observationstatus.WebHome]](e.g. estimate, provisional, normal)186 +* Decimals (i.e. number of decimal digits used in numerical observations) 187 +* Observation Status (e.g. estimate, provisional, normal) 178 178 179 -Moreover, additional [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]may be considered as mandatory when a specific[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]] is defined.189 +Moreover, additional attributes may be considered as mandatory when a specific data structure definition is defined. 180 180 181 -**//Avoid creating a new [[code list>>doc:sdmx:Glossary.Codelist.WebHome]]where one already exists.//** It is highly recommended that structural definitions and[[code lists>>doc:sdmx:Glossary.Codelist.WebHome]]beconsistent with internationally agreed standard methodologies, wherever they exist, e.g., System of National Accounts 1993; Balance of Payments Manual, Fifth Edition; Monetary and Financial Statistics Manual; Government Finance Statistics Manual, etc. When setting-up a new data exchange, the following order of priority is suggested when considering the use of[[code lists>>doc:sdmx:Glossary.Code list.WebHome]]:191 +**//Avoid creating a new code list where one already exists.//** It is highly recommended that structural definitions and code lists be consistent with internationally agreed standard methodologies, wherever they exist, e.g., System of National Accounts 1993; Balance of Payments Manual, Fifth Edition; Monetary and Financial Statistics Manual; Government Finance Statistics Manual, etc. When setting-up a new data exchange, the following order of priority is suggested when considering the use of code lists: 182 182 183 -* international standard [[code lists>>doc:sdmx:Glossary.Code list.WebHome]];184 -* international [[code lists>>doc:sdmx:Glossary.Codelist.WebHome]] supplemented by other international and/or regional institutions;193 +* international standard code lists; 194 +* international code lists supplemented by other international and/or regional institutions; 185 185 * standardised lists used already by international institutions; 186 -* new [[code lists>>doc:sdmx:Glossary.Codelist.WebHome]]agreed between two international or regional institutions;187 -* new specific [[code lists>>doc:sdmx:Glossary.Code list.WebHome]].196 +* new code lists agreed between two international or regional institutions; 197 +* new specific code lists. 188 188 189 -The same [[code list>>doc:sdmx:Glossary.Codelist.WebHome]]can be used for several statistical(% style="color:#e74c3c" %)concepts(%%), within a[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]] or across DSDs. Note that[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] has recognised that these classifications are often quite large and the usage of[[codes>>doc:sdmx:Glossary.Code.WebHome]]in any one[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] is only a small extract of the full[[code list>>doc:sdmx:Glossary.Codelist.WebHome]].In this(% style="color:#e74c3c" %)version(%%)of the standard it is possible to exchange and disseminate a **partial[[code list>>doc:sdmx:Glossary.Code list.WebHome]]** which is extracted from the full[[code list>>doc:sdmx:Glossary.Codelist.WebHome]]and which supports the[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]values valid for a particular[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].199 +The same code list can be used for several statistical concepts, within a data structure definition or across DSDs. Note that SDMX has recognised that these classifications are often quite large and the usage of codes in any one DSD is only a small extract of the full code list. In this version of the standard it is possible to exchange and disseminate a **partial code list** which is extracted from the full code list and which supports the dimension values valid for a particular DSD. 190 190 191 - __Data Structure Definition Structure__201 +=== Data Structure Definition Structure === 192 192 193 -The following items have to be specified by a structural definitions [[maintenance agency>>doc:sdmx:Glossary.Maintenanceagency.WebHome]]when defining a new[[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]]:203 +The following items have to be specified by a structural definitions maintenance agency when defining a new data structure definition: 194 194 195 - [[Data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]]([[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]) identification:205 +Data structure definition (DSD) identification: 196 196 197 -* [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] identifier198 -* [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] name207 +* DSD identifier 208 +* DSD name 199 199 200 -A list of metadata (% style="color:#e74c3c" %)concepts(%%)assigned as[[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]of the[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]].For each:210 +A list of metadata concepts assigned as dimensions of the data structure definition. For each: 201 201 202 -* (statistical) (% style="color:#e74c3c" %)concept(%%)identifier203 -* ordinal number of the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]in the key structure ([[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]only)204 -* [[code list>>doc:sdmx:Glossary.Codelist.WebHome]] identifier (Id,(% style="color:#e74c3c" %)version(%%),[[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]]) if the[[representation>>doc:sdmx:Glossary.Representation.WebHome]]is coded212 +* (statistical) concept identifier 213 +* ordinal number of the dimension in the key structure (SDMX-EDI only) 214 +* code list identifier (Id, version, maintenance agency) if the representation is coded 205 205 206 -A list of (statistical) (% style="color:#e74c3c" %)concepts(%%)assigned as[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]for the[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]].For each:216 +A list of (statistical) concepts assigned as attributes for the data structure definition. For each: 207 207 208 -* (statistical) (% style="color:#e74c3c" %)concept(%%)identifier209 -* [[code list>>doc:sdmx:Glossary.Codelist.WebHome]] identifier if the(% style="color:#e74c3c" %)concept(%%)is coded218 +* (statistical) concept identifier 219 +* code list identifier if the concept is coded 210 210 * assignment status: mandatory or conditional 211 -* attachment (% style="color:#e74c3c" %)level212 -* maximum text length for the uncoded (% style="color:#e74c3c" %)concepts213 -* maximum [[code>>doc:sdmx:Glossary.Code.WebHome]]length for the coded(% style="color:#e74c3c" %)concepts221 +* attachment level 222 +* maximum text length for the uncoded concepts 223 +* maximum code length for the coded concepts 214 214 215 -A list of the [[code lists>>doc:sdmx:Glossary.Codelist.WebHome]]used in the[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]].For each:225 +A list of the code lists used in the data structure definition. For each: 216 216 217 -* [[code list>>doc:sdmx:Glossary.Codelist.WebHome]] identifier218 -* [[code list>>doc:sdmx:Glossary.Codelist.WebHome]]name219 -* [[code>>doc:sdmx:Glossary.Code.WebHome]]values and descriptions227 +* code list identifier 228 +* code list name 229 +* code values and descriptions 220 220 221 -Definition of [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]definitions. Two (or more) partners performing data exchanges in a certain context need to agree on:231 +Definition of data flow definitions. Two (or more) partners performing data exchanges in a certain context need to agree on: 222 222 223 -* the list of [[data set>>doc:sdmx:Glossary.Dataset.WebHome]]identifiers they will be using;224 -* for each [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]:233 +* the list of data set identifiers they will be using; 234 +* for each data flow: 225 225 * its content and description 226 -* the relevant [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] that defines the structure of the data reported or disseminated according the the[[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]definition236 +* the relevant DSD that defines the structure of the data reported or disseminated according the the dataflow definition 227 227 228 - ====3.4.1.3 Exchanging Attributes====238 +**3.4.1.3 Exchanging Attributes** 229 229 230 - =====//3.4.1.3.1 Attributes on series, sibling and data set level //=====240 +**//3.4.1.3.1 Attributes on series, sibling and data set level //**//Static properties//. 231 231 232 -//Static properties//. 242 +* Upon creation of a series the sender has to provide to the receiver values for all mandatory attributes. In case they are available, values for conditional attributes should also be provided. Whereas initially this information may be provided by means other than SDMX-ML or SDMX-EDI messages (e.g. paper, telephone) it is expected that partner institutions will be in a position to provide this information in SDMX-ML or SDMX-EDI format over time. 243 +* A centre may agree with its data exchange partners special procedures for authorising the setting of attributes' initial values. 244 +* Attribute values at a data set level are set and maintained exclusively by the centre administrating the exchanged data set. 233 233 234 -* Upon creation of a series the sender has to provide to the receiver values for all mandatory [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]. In case they are available, values for conditional [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] should also be provided. Whereas initially this information may be provided by means other than [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] or [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]] messages (e.g. paper, telephone) it is expected that partner institutions will be in a position to provide this information in [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] or [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]] format over time. 235 -* A centre may agree with its data exchange partners special procedures for authorising the setting of [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]' initial values. 236 -* [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]] values at a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] (% style="color:#e74c3c" %)level(%%) are set and maintained exclusively by the centre administrating the exchanged [[data set>>doc:sdmx:Glossary.Data set.WebHome]]. 237 - 238 238 //Communication of changes// to the centre. 239 239 240 -* Following the creation of a series, the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]values do not have to be reported again by senders, as long as they do not change.241 -* Whenever changes in [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]values for a series (or[[sibling group>>doc:sdmx:Glossary.Sibling group.WebHome]]) occur, the reporting institutions should report either all[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]values again (this is the recommended option) or only the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]values which have changed. This applies both to the mandatory and the conditional[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]].For example, if a previously reported value for a conditional[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]is no longer valid, this has to be reported to the centre.242 -* A centre may agree with its data exchange partners special procedures for authorising modifications in the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]values.248 +* Following the creation of a series, the attribute values do not have to be reported again by senders, as long as they do not change. 249 +* Whenever changes in attribute values for a series (or sibling group) occur, the reporting institutions should report either all attribute values again (this is the recommended option) or only the attribute values which have changed. This applies both to the mandatory and the conditional attributes. For example, if a previously reported value for a conditional attribute is no longer valid, this has to be reported to the centre. 250 +* A centre may agree with its data exchange partners special procedures for authorising modifications in the attribute values. 243 243 244 -Communication of observation (% style="color:#e74c3c" %)level(%%)[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]“[[observation status>>doc:sdmx:Glossary.Observation status.WebHome]]”, "observation[[confidentiality>>doc:sdmx:Glossary.Confidentiality.WebHome]]", "observation pre-break".252 +Communication of observation level attributes “observation status”, "observation confidentiality", "observation pre-break". 245 245 246 -* In [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]], the observation(% style="color:#e74c3c" %)level(%%)[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]“[[observation status>>doc:sdmx:Glossary.Observation status.WebHome]]” is part of the fixed syntax of the ARR segment used for observation reporting. Whenever an observation is exchanged, the corresponding[[observation status>>doc:sdmx:Glossary.Observationstatus.WebHome]] must also be exchanged attached to the observation, regardless of whether it has changed or not since the previous data exchange. This rule also applies to the use of the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]formats, although the syntax does not necessarily require this.247 -* If the “ [[observation status>>doc:sdmx:Glossary.Observation status.WebHome]]” changes and the observation remains unchanged, both[[components>>doc:sdmx:Glossary.Component.WebHome]]would have to be reported.248 -* For [[Data Structure Definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]having also the observation(% style="color:#e74c3c" %)level(%%)[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]“observation[[confidentiality>>doc:sdmx:Glossary.Confidentiality.WebHome]]” and "observation pre-break" defined, this rule applies to these[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]as well: if an institution receives from another institution an observation with an[[observation status>>doc:sdmx:Glossary.Observationstatus.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]only attached, this means that the associated observation[[confidentiality>>doc:sdmx:Glossary.Confidentiality.WebHome]]and prebreak observation[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]either never existed or from now they do not have a value for this observation.254 +* In SDMX-EDI, the observation level attribute “observation status” is part of the fixed syntax of the ARR segment used for observation reporting. Whenever an observation is exchanged, the corresponding observation status must also be exchanged attached to the observation, regardless of whether it has changed or not since the previous data exchange. This rule also applies to the use of the SDMX-ML formats, although the syntax does not necessarily require this. 255 +* If the “observation status” changes and the observation remains unchanged, both components would have to be reported. 256 +* For Data Structure Definitions having also the observation level attributes “observation confidentiality” and "observation pre-break" defined, this rule applies to these attribute as well: if an institution receives from another institution an observation with an observation status attribute only attached, this means that the associated observation confidentiality and prebreak observation attributes either never existed or from now they do not have a value for this observation. 249 249 250 -=== 3.4.2 Best Practices for Batch Data Exchange === 258 +==== 3.4.2 Best Practices for Batch Data Exchange ==== 251 251 252 - ====3.4.2.1 Introduction====260 +**3.4.2.1 Introduction** 253 253 254 -Batch data exchange is the exchange and maintenance of entire databases between counterparties. It is an activity that often employs [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]formats, and might also use the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]][[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]-specific[[data set>>doc:sdmx:Glossary.Dataset.WebHome]].The following points apply equally to both formats.262 +Batch data exchange is the exchange and maintenance of entire databases between counterparties. It is an activity that often employs SDMX-EDI formats, and might also use the SDMX-ML DSD-specific data set. The following points apply equally to both formats. 255 255 256 - ====3.4.2.2 Positioning of the Dimension "Frequency"====264 +**3.4.2.2 Positioning of the Dimension "Frequency"** 257 257 258 -The position of the “frequency” [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]is unambiguously identified in the[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]].Moreover, most central institutions devising structural definitions have decided to assign to this[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]the first position in the key structure. This facilitates the easy identification of this[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]], something that it is necessary to frequency's crucial role in several database systems and in attaching[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]at the “sibling” group(% style="color:#e74c3c" %)level(%%).266 +The position of the “frequency” dimension is unambiguously identified in the data structure definition. Moreover, most central institutions devising structural definitions have decided to assign to this dimension the first position in the key structure. This facilitates the easy identification of this dimension, something that it is necessary to frequency's crucial role in several database systems and in attaching attributes at the “sibling” group level. 259 259 260 - ====3.4.2.3 Identification of Data Structure Definitions (DSDs)====268 +**3.4.2.3 Identification of Data Structure Definitions (DSDs)** 261 261 262 -In order to facilitate the easy and immediate recognition of the structural definition [[maintenance agency>>doc:sdmx:Glossary.Maintenanceagency.WebHome]]that defined a[[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]], most central institutions devising structural definitions use the first characters of the[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]] identifiers to identify their institution: e.g. BIS_EER, EUROSTAT_BOP_01, ECB_BOP1, etc.270 +In order to facilitate the easy and immediate recognition of the structural definition maintenance agency that defined a data structure definition, most central institutions devising structural definitions use the first characters of the data structure definition identifiers to identify their institution: e.g. BIS_EER, EUROSTAT_BOP_01, ECB_BOP1, etc. 263 263 264 - ====3.4.2.4 Identification of the Data Flows====272 +**3.4.2.4 Identification of the Data Flows** 265 265 266 -In order to facilitate the easy and immediate recognition of the institution administrating a [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]definitions, many central institutions prefer to use the first characters of the[[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]definition identifiers to identify their institution: e.g. BIS_EER, ECB_BOP1, ECB_BOP1, etc. Note that in GESMES/TS the[[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]plays the role of the[[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]definition (see //[[DataSet>>doc:sdmx:Glossary.Dataset.WebHome]]//in the[[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]//)//.274 +In order to facilitate the easy and immediate recognition of the institution administrating a data flow definitions, many central institutions prefer to use the first characters of the data flow definition identifiers to identify their institution: e.g. BIS_EER, ECB_BOP1, ECB_BOP1, etc. Note that in GESMES/TS the Data Set plays the role of the data flow definition (see //DataSet //in the SDMX-IM//)//. 267 267 268 -The statistical information in [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]is broken down into two fundamental parts -[[structural metadata>>doc:sdmx:Glossary.Structuralmetadata.WebHome]](comprising the[[Data Structure Definition>>doc:sdmx:Glossary.Data structure definition.WebHome]], and associated(% style="color:#e74c3c" %)Concepts(%%)and[[Code Lists>>doc:sdmx:Glossary.Code list.WebHome]]) - see Framework for Standards -, and observational data (the[[DataSet>>doc:sdmx:Glossary.Data set.WebHome]]). This is an important distinction, with specific terminology associated with each part. Data - which is typically a set of numeric observations at specific points in time - is organized into[[data sets>>doc:sdmx:Glossary.Dataset.WebHome]](//[[DataSet>>doc:sdmx:Glossary.Data set.WebHome]]//) These[[data sets>>doc:sdmx:Glossary.Dataset.WebHome]]are structured according to a specific[[Data Structure Definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]](//DataStructureDefinition//) and are described in the[[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]definition (//DataflowDefinition)// The[[Data Structure Definition>>doc:sdmx:Glossary.Datastructuredefinition.WebHome]] describes the metadata that allows an understanding of what is expressed in the[[data set>>doc:sdmx:Glossary.Data set.WebHome]], whilst the[[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]]definition provides the identifier and other important information (such as the periodicity of reporting) that is common to all of its[[component>>doc:sdmx:Glossary.Component.WebHome]][[data sets>>doc:sdmx:Glossary.Dataset.WebHome]].276 +The statistical information in SDMX is broken down into two fundamental parts - structural metadata (comprising the Data Structure Definition, and associated Concepts and Code Lists) - see Framework for Standards -, and observational data (the DataSet). This is an important distinction, with specific terminology associated with each part. Data - which is typically a set of numeric observations at specific points in time - is organized into data sets (//DataSet//) These data sets are structured according to a specific Data Structure Definition (//DataStructureDefinition//) and are described in the data flow definition (//DataflowDefinition)// The Data Structure Definition describes the metadata that allows an understanding of what is expressed in the data set, whilst the data flow definition provides the identifier and other important information (such as the periodicity of reporting) that is common to all of its component data sets. 269 269 270 -Note that the role of the [[Data Flow>>doc:sdmx:Glossary.Dataflow.WebHome]](called //DataflowDefintion// in the model) and[[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]is very specific in the model, and the terminology used may not be the same as used in all organisations, and specifically the term[[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]is used differently in[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]thanin GESMES/TS. Essentially the GESMES/TS term[[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]is, in[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], the[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]Definition" whist the term[[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]in[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]is used to describe the "container" for an instance of the data.278 +Note that the role of the Data Flow (called //DataflowDefintion// in the model) and Data Set is very specific in the model, and the terminology used may not be the same as used in all organisations, and specifically the term Data Set is used differently in SDMX than in GESMES/TS. Essentially the GESMES/TS term "Data Set" is, in SDMX, the "Dataflow Definition" whist the term "Data Set" in SDMX is used to describe the "container" for an instance of the data. 271 271 272 - ====3.4.2.5 Special Issues====280 +**3.4.2.5 Special Issues** 273 273 274 274 ===== 3.4.2.5.1 "Frequency" related issues ===== 275 275 276 276 **//Special frequencies.//** The issue of data collected at special (regular or irregular) intervals at a lower than daily frequency (e.g. 24 or 36 or 48 observations per year, on irregular days during the year) is not extensively discussed here. However, for data exchange purposes: 277 277 278 -* such data can be (% style="color:#e74c3c" %)mapped(%%)into a series with daily frequency; this daily series will only hold observations for those days on which the measured event takes place;279 -* if the collection intervals are regular, additional values to the existing frequency [[code list>>doc:sdmx:Glossary.Code list.WebHome]](s) could be added in the future.286 +* such data can be mapped into a series with daily frequency; this daily series will only hold observations for those days on which the measured event takes place; 287 +* if the collection intervals are regular, additional values to the existing frequency code list(s) could be added in the future. 280 280 281 -**//Tick data.//** The issue of data collected at irregular intervals at a higher than daily frequency (e.g. tick-by-tick data) is not discussed here either. However, for data exchange purposes, such series can already be exchanged in the [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]format by using the option to send observations with the associated time stamp.289 +**//Tick data.//** The issue of data collected at irregular intervals at a higher than daily frequency (e.g. tick-by-tick data) is not discussed here either. However, for data exchange purposes, such series can already be exchanged in the SDMX-EDI format by using the option to send observations with the associated time stamp. 282 282 291 + 283 283 = 4 General Notes for Implementers = 284 284 285 -This section discusses a number of topics other than the exchange of [[data sets>>doc:sdmx:Glossary.Dataset.WebHome]]in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]and[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]].Supported only in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]], these topics include the use of the[[reference metadata>>doc:sdmx:Glossary.Referencemetadata.WebHome]] mechanism in[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], the use of[[Structure Sets>>doc:sdmx:Glossary.Structureset.WebHome]]and[[Reporting Taxonomies>>doc:sdmx:Glossary.Reporting taxonomy.WebHome]], the use of Processes, a discussion of time and data-typing, and some of the conventional mechanisms within the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]Structure message regarding versioning and external referencing.294 +This section discusses a number of topics other than the exchange of data sets in SDMX-ML and SDMX-EDI. Supported only in SDMX-ML, these topics include the use of the reference metadata mechanism in SDMX, the use of Structure Sets and Reporting Taxonomies, the use of Processes, a discussion of time and data-typing, and some of the conventional mechanisms within the SDMX-ML Structure message regarding versioning and external referencing. 286 286 287 287 This section does not go into great detail on these topics, but provides a useful overview of these features to assist implementors in further use of the parts of the specification which are relevant to them. 288 288 289 289 == 4.1 Representations == 290 290 291 -There are several different [[representations>>doc:sdmx:Glossary.Representation.WebHome]]in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]], taken from XML Schemas and common programming languages. The table below describes the various[[representations>>doc:sdmx:Glossary.Representation.WebHome]]which are found in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]], and their equivalents.300 +There are several different representations in SDMX-ML, taken from XML Schemas and common programming languages. The table below describes the various representations which are found in SDMX-ML, and their equivalents. 292 292 293 -(% style="width:912.294px" %) 294 -|(% style="width:172px" %)**SDMX-ML Data Type**|(% style="width:204px" %)**XML Schema Data Type**|(% style="width:189px" %)**.NET Framework Type**|(% style="width:342px" %)((( 295 -**Java Data Type ** 302 +|**SDMX-ML Data Type**|**XML Schema Data Type**|**.NET Framework Type**|((( 303 +**Java Data Type** 304 + 305 +**~ ** 296 296 ))) 297 -|(% style="width:172px" %)String|(% style="width:204px" %)xsd:string|(% style="width:189px" %)System.String|(% style="width:342px" %)java.lang.String 298 -|(% style="width:172px" %)Big Integer|(% style="width:204px" %)xsd:integer|(% style="width:189px" %)System.Decimal|(% style="width:342px" %)java.math.BigInteg er 299 -|(% style="width:172px" %)Integer|(% style="width:204px" %)xsd:int|(% style="width:189px" %)System.Int32|(% style="width:342px" %)int 300 -|(% style="width:172px" %)Long|(% style="width:204px" %)xsd.long|(% style="width:189px" %)System.Int64|(% style="width:342px" %)long 301 -|(% style="width:172px" %)Short|(% style="width:204px" %)xsd:short|(% style="width:189px" %)System.Int16|(% style="width:342px" %)short 302 -|(% style="width:172px" %)Decimal|(% style="width:204px" %)xsd:decimal|(% style="width:189px" %)System.Decimal|(% style="width:342px" %)java.math.BigDecim al 303 -|(% style="width:172px" %)Float|(% style="width:204px" %)xsd:float|(% style="width:189px" %)System.Single|(% style="width:342px" %)float 304 -|(% style="width:172px" %)Double|(% style="width:204px" %)xsd:double|(% style="width:189px" %)System.Double|(% style="width:342px" %)double 305 -|(% style="width:172px" %)Boolean|(% style="width:204px" %)xsd:boolean|(% style="width:189px" %)System.Boolean|(% style="width:342px" %)boolean 306 -|(% style="width:172px" %)URI|(% style="width:204px" %)xsd:anyURI|(% style="width:189px" %)System.Uri|(% style="width:342px" %)Java.net.URI or java.lang.String 307 -|(% style="width:172px" %)DateTime|(% style="width:204px" %)xsd:dateTime|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 308 -|(% style="width:172px" %)Time|(% style="width:204px" %)xsd:time|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 309 -|(% style="width:172px" %)GregorianYear|(% style="width:204px" %)xsd:gYear|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 310 -|(% style="width:172px" %)GregorianMonth|(% style="width:204px" %)xsd:gYearMonth|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 311 -|(% style="width:172px" %)GregorianDay|(% style="width:204px" %)xsd:date|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 312 -|(% style="width:172px" %)((( 313 -Day, MonthDay, Month 314 -)))|(% style="width:204px" %)xsd:g*|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 315 -|(% style="width:172px" %)Duration|(% style="width:204px" %)xsd:duration |(% style="width:189px" %)System.TimeSpa|(% style="width:342px" %)javax.xml.datatype 316 -|(% style="width:172px" %) |(% style="width:204px" %) |(% style="width:189px" %)n|(% style="width:342px" %).Duration 307 +|String|xsd:string|System.String|java.lang.String 308 +|Big Integer|xsd:integer|System.Decimal|java.math.BigInteg er 309 +|Integer|xsd:int|System.Int32|int 310 +|Long|xsd.long|System.Int64|long 311 +|Short|xsd:short|System.Int16|short 312 +|Decimal|xsd:decimal|System.Decimal|java.math.BigDecim al 313 +|Float|xsd:float|System.Single|float 314 +|Double|xsd:double|System.Double|double 315 +|Boolean|xsd:boolean|System.Boolean|boolean 316 +|URI|xsd:anyURI|System.Uri|Java.net.URI or java.lang.String 317 +|DateTime|xsd:dateTime|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 318 +|Time|xsd:time|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 319 +|GregorianYear|xsd:gYear|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 320 +|GregorianMont h|xsd:gYearMont h|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 321 +|GregorianDay|xsd:date|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 322 +|((( 323 +Day, 317 317 318 -There are also a number of [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] data types which do not have these direct correspondences, often because they are composite [[representations>>doc:sdmx:Glossary.Representation.WebHome]] or restrictions of a broader data type. For most of these, there are simple types which can be referenced from the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] schemas, for others a derived simple type will be necessary: 325 +MonthDay, Month 326 +)))|xsd:g*|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 327 +|Duration|xsd:duration |System.TimeSpa|javax.xml.datatype 328 +|**SDMX-ML Data Type**|**XML Schema Data Type**|**.NET Framework Type**|((( 329 +**Java Data Type** 319 319 331 +**~ ** 332 +))) 333 +| | |n|.Duration 334 + 335 +There are also a number of SDMX-ML data types which do not have these direct correspondences, often because they are composite representations or restrictions of a broader data type. For most of these, there are simple types which can be referenced from the SDMX schemas, for others a derived simple type will be necessary: 336 + 320 320 * AlphaNumeric (common:AlphaNumericType, string which only allows A-z and 0-9) 321 321 * Alpha (common:AlphaType, string which only allows A-z) 322 322 * Numeric (common:NumericType, string which only allows 0-9, but is not numeric so that is can having leading zeros) 323 323 * Count (xs:integer, a sequence with an interval of “1”) 324 -* InclusiveValueRange (xs:decimal with the minValue and maxValue [[facets>>doc:sdmx:Glossary.Facet.WebHome]]supplying the bounds)325 -* ExclusiveValueRange (xs:decimal with the minValue and maxValue [[facets>>doc:sdmx:Glossary.Facet.WebHome]]supplying the bounds)341 +* InclusiveValueRange (xs:decimal with the minValue and maxValue facets supplying the bounds) 342 +* ExclusiveValueRange (xs:decimal with the minValue and maxValue facets supplying the bounds) 326 326 * Incremental (xs:decimal with a specified interval; the interval is typically enforced outside of the XML validation) 327 327 * TimeRange (common:TimeRangeType, start DateTime + Duration,) 328 -* ObservationalTimePeriod (common: ObservationalTimePeriodType, a union of StandardTimePeriod and TimeRange). 345 +* ObservationalTimePeriod (common: ObservationalTimePeriodType, a union of StandardTimePeriod and TimeRange). 329 329 * StandardTimePeriod (common: StandardTimePeriodType, a union of BasicTimePeriod and TimeRange). 330 330 * BasicTimePeriod (common: BasicTimePeriodType, a union of GregorianTimePeriod and DateTime) 331 331 * GregorianTimePeriod (common:GregorianTimePeriodType, a union of GregorianYear, GregorianMonth, and GregorianDay) ... ... @@ -340,30 +340,32 @@ 340 340 * KeyValues (common:DataKeyType) 341 341 * IdentifiableReference (types for each identifiable object) 342 342 * DataSetReference (common:DataSetReferenceType) 343 -* AttachmentConstraintReference (common:AttachmentConstraintReferenceType)360 +* AttachmentConstraintReference 344 344 345 - Datatypesalsohavea setof [[facets>>doc:sdmx:Glossary.Facet.WebHome]]:362 +(common:AttachmentConstraintReferenceType) 346 346 364 +Data types also have a set of facets: 365 + 347 347 * isSequence = true | false (indicates a sequentially increasing value) 348 348 * minLength = positive integer (# of characters/digits) 349 349 * maxLength = positive integer (# of characters/digits) 350 -* startValue = [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]](for numeric sequence)351 -* endValue = [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]](for numeric sequence)352 -* interval = [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]](for numeric sequence)369 +* startValue = decimal (for numeric sequence) 370 +* endValue = decimal (for numeric sequence) 371 +* interval = decimal (for numeric sequence) 353 353 * timeInterval = duration 354 354 * startTime = BasicTimePeriod (for time range) 355 355 * endTime = BasicTimePeriod (for time range) 356 -* minValue = [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]](for numeric range)357 -* maxValue = [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]](for numeric range)358 -* [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]]= Integer (# of digits to right of[[decimal>>doc:sdmx:Glossary.Decimals.WebHome]]point)375 +* minValue = decimal (for numeric range) 376 +* maxValue = decimal (for numeric range) 377 +* decimal = Integer (# of digits to right of decimal point) 359 359 * pattern = (a regular expression, as per W3C XML Schema) 360 360 * isMultiLingual = boolean (for specifying text can occur in more than one language) 361 361 362 -Note that [[code lists>>doc:sdmx:Glossary.Codelist.WebHome]] may also have textual[[representations>>doc:sdmx:Glossary.Representation.WebHome]]assigned to them, in addition to their enumeration of codes.s381 +Note that code lists may also have textual representations assigned to them, in addition to their enumeration of codes.s 363 363 364 364 == 4.2 Time and Time Format == 365 365 366 -=== 4.2.1 Introduction === 385 +==== 4.2.1 Introduction ==== 367 367 368 368 First, it is important to recognize that most observation times are a period. SDMX specifies precisely how Time is handled. 369 369 ... ... @@ -371,47 +371,50 @@ 371 371 372 372 The hierarchy of time formats is as follows (**bold** indicates a category which is made up of multiple formats, //italic// indicates a distinct format): 373 373 374 -* **Observational Time Period** 375 -** **Standard Time Period** 376 -*** **Basic Time Period** 377 -**** **Gregorian Time Period** 378 -**** //Date Time// 379 -*** **Reporting Time Period** 380 -** //Time Range// 393 +* **Observational Time Period **o **Standard Time Period** 381 381 395 + § **Basic Time Period** 396 + 397 +* **Gregorian Time Period** 398 +* //Date Time// 399 + 400 +§ **Reporting Time Period **o //Time Range// 401 + 382 382 The details of these time period categories and of the distinct formats which make them up are detailed in the sections to follow. 383 383 384 -=== 4.2.2 Observational Time Period === 404 +==== 4.2.2 Observational Time Period ==== 385 385 386 -This is the superset of all time [[representations>>doc:sdmx:Glossary.Representation.WebHome]]in[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]].This allows for time to be expressed as any of the allowable formats.406 +This is the superset of all time representations in SDMX. This allows for time to be expressed as any of the allowable formats. 387 387 388 -=== 4.2.3 Standard Time Period === 408 +==== 4.2.3 Standard Time Period ==== 389 389 390 -This is the superset of any predefined [[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]] or a distinct point in time. A[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]]consists of a distinct start and end point. If the start and end of a period are expressed as date instead of a complete date time, then it is implied that the start of the period is the beginning of the start day (i.e. 00:00:00) and the end of the period is the end of the end day (i.e. 23:59:59).410 +This is the superset of any predefined time period or a distinct point in time. A time period consists of a distinct start and end point. If the start and end of a period are expressed as date instead of a complete date time, then it is implied that the start of the period is the beginning of the start day (i.e. 00:00:00) and the end of the period is the end of the end day (i.e. 23:59:59). 391 391 392 -=== 4.2.4 Gregorian Time Period === 412 +==== 4.2.4 Gregorian Time Period ==== 393 393 394 -A Gregorian [[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]] is always represented by a Gregorian year, year-month, or day. These are all based on ISO 8601 dates. The[[representation>>doc:sdmx:Glossary.Representation.WebHome]]in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]messages and the period covered by each of the Gregorian[[time periods>>doc:sdmx:Glossary.Timeperiod.WebHome]]are as follows:414 +A Gregorian time period is always represented by a Gregorian year, year-month, or day. These are all based on ISO 8601 dates. The representation in SDMX-ML messages and the period covered by each of the Gregorian time periods are as follows: 395 395 396 -**Gregorian Year:** 416 +**Gregorian Year:** 417 + 397 397 Representation: xs:gYear (YYYY) 398 -Period: the start of January 1 to the end of December 31 399 399 400 -**Gregorian Year Month**: 420 +Period: the start of January 1 to the end of December 31 **Gregorian Year Month**: 421 + 401 401 Representation: xs:gYearMonth (YYYY-MM) 402 -Period: the start of the first day of the month to end of the last day of the month 403 403 404 -**Gregorian Day**: 424 +Period: the start of the first day of the month to end of the last day of the month **Gregorian Day**: 425 + 405 405 Representation: xs:date (YYYY-MM-DD) 427 + 406 406 Period: the start of the day (00:00:00) to the end of the day (23:59:59) 407 407 408 -=== 4.2.5 Date Time === 430 +==== 4.2.5 Date Time ==== 409 409 410 -This is used to unambiguously state that a date-time represents an observation at a single point in time. Therefore, if one wants to use [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]for data which is measured at a distinct point in time rather than being reported over a period, the date-time[[representation>>doc:sdmx:Glossary.Representation.WebHome]]can be used.432 +This is used to unambiguously state that a date-time represents an observation at a single point in time. Therefore, if one wants to use SDMX for data which is measured at a distinct point in time rather than being reported over a period, the date-time representation can be used. 411 411 412 - [[Representation>>doc:sdmx:Glossary.Representation.WebHome]]:xs:dateTime (YYYY-MM-DDThh:mm:ss)[[(% class="wikiinternallink wikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[1~]^^>>path:#_ftn1]]434 +Representation: xs:dateTime (YYYY-MM-DDThh:mm:ss)[[(% class="wikiinternallink wikiinternallink" %)^^~[1~]^^>>path:#_ftn1]] 413 413 414 -=== 4.2.6 Standard Reporting Period === 436 +==== 4.2.6 Standard Reporting Period ==== 415 415 416 416 Standard reporting periods are periods of time in relation to a reporting year. Each of these standard reporting periods has a duration (based on the ISO 8601 definition) associated with it. The general format of a reporting period is as follows: 417 417 ... ... @@ -418,52 +418,75 @@ 418 418 [REPORTING_YEAR]-[PERIOD_INDICATOR][PERIOD_VALUE] 419 419 420 420 Where: 443 + 421 421 REPORTING_YEAR represents the reporting year as four digits (YYYY) PERIOD_INDICATOR identifies the type of period which determines the duration of the period 445 + 422 422 PERIOD_VALUE indicates the actual period within the year 423 423 424 -The following section details each of the standard reporting periods defined in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]:448 +The following section details each of the standard reporting periods defined in SDMX: 425 425 426 -**Reporting Year**: 427 -Period Indicator: A 450 +**Reporting Year**: 451 + 452 + Period Indicator: A 453 + 428 428 Period Duration: P1Y (one year) 455 + 429 429 Limit per year: 1 430 -Representation: common:ReportingYearType (YYYY-A1, e.g. 2000-A1) 431 431 432 -**Reporting Semester:** 433 -Period Indicator: S 458 +Representation: common:ReportingYearType (YYYY-A1, e.g. 2000-A1) **Reporting Semester:** 459 + 460 + Period Indicator: S 461 + 434 434 Period Duration: P6M (six months) 463 + 435 435 Limit per year: 2 436 -Representation: common:ReportingSemesterType (YYYY-Ss, e.g. 2000-S2) 437 437 438 -**Reporting Trimester:** 439 -Period Indicator: T 466 +Representation: common:ReportingSemesterType (YYYY-Ss, e.g. 2000-S2) **Reporting Trimester:** 467 + 468 + Period Indicator: T 469 + 440 440 Period Duration: P4M (four months) 471 + 441 441 Limit per year: 3 442 -Representation: common:ReportingTrimesterType (YYYY-Tt, e.g. 2000-T3) 443 443 444 -**Reporting Quarter:** 445 -Period Indicator: Q 474 +Representation: common:ReportingTrimesterType (YYYY-Tt, e.g. 2000-T3) **Reporting Quarter:** 475 + 476 + Period Indicator: Q 477 + 446 446 Period Duration: P3M (three months) 479 + 447 447 Limit per year: 4 448 -Representation: common:ReportingQuarterType (YYYY-Qq, e.g. 2000-Q4) 449 449 450 -**Reporting Month**: 482 +Representation: common:ReportingQuarterType (YYYY-Qq, e.g. 2000-Q4) **Reporting Month**: 483 + 451 451 Period Indicator: M 485 + 452 452 Period Duration: P1M (one month) 487 + 453 453 Limit per year: 1 489 + 454 454 Representation: common:ReportingMonthType (YYYY-Mmm, e.g. 2000-M12) Notes: The reporting month is always represented as two digits, therefore 1-9 are 0 padded (e.g. 01). This allows the values to be sorted chronologically using textual sorting methods. 455 455 456 456 **Reporting Week**: 493 + 457 457 Period Indicator: W 495 + 458 458 Period Duration: P7D (seven days) 497 + 459 459 Limit per year: 53 499 + 460 460 Representation: common:ReportingWeekType (YYYY-Www, e.g. 2000-W53) 461 -Notes: There are either 52 or 53 weeks in a reporting year. This is based on the ISO 8601 definition of a week (Monday - Saturday), where the first week of a reporting year is defined as the week with the first Thursday on or after the reporting year start day.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[2~]^^>>path:#_ftn2]](%%) The reporting week is always represented as two digits, therefore 1-9 are 0 padded (e.g. 01). This allows the values to be sorted chronologically using textual sorting methods. 462 462 502 +Notes: There are either 52 or 53 weeks in a reporting year. This is based on the ISO 8601 definition of a week (Monday - Saturday), where the first week of a reporting year is defined as the week with the first Thursday on or after the reporting year start day.[[(% class="wikiinternallink wikiinternallink" %)^^~[2~]^^>>path:#_ftn2]](%%) The reporting week is always represented as two digits, therefore 1-9 are 0 padded (e.g. 01). This allows the values to be sorted chronologically using textual sorting methods. 503 + 463 463 **Reporting Day**: 505 + 464 464 Period Indicator: D 507 + 465 465 Period Duration: P1D (one day) 509 + 466 466 Limit per year: 366 511 + 467 467 Representation: common:ReportingDayType (YYYY-Dddd, e.g. 2000-D366) Notes: There are either 365 or 366 days in a reporting year, depending on whether the reporting year includes leap day (February 29). The reporting day is always represented as three digits, therefore 1-99 are 0 padded (e.g. 001). 468 468 469 469 This allows the values to be sorted chronologically using textual sorting methods. ... ... @@ -470,116 +470,149 @@ 470 470 471 471 The meaning of a reporting year is always based on the start day of the year and requires that the reporting year is expressed as the year at the start of the period. This start day is always the same for a reporting year, and is expressed as a day and a month (e.g. July 1). Therefore, the reporting year 2000 with a start day of July 1 begins on July 1, 2000. 472 472 473 -A specialized [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]](reporting year start day) exists for the purpose of communicating the reporting year start day. This[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]has a fixed identifier (REPORTING_YEAR_START_DAY) and a fixed[[representation>>doc:sdmx:Glossary.Representation.WebHome]](xs:gMonthDay) so that it can always be easily identified and processed in a data message. Although this[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]exists in specialized sub-class, it functions the same as any other[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]outside of its identification and[[representation>>doc:sdmx:Glossary.Representation.WebHome]].It must takes its identity from a(% style="color:#e74c3c" %)concept(%%)and state its relationship with other[[components>>doc:sdmx:Glossary.Component.WebHome]]of the[[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]].The ability to state this relationship allows this reporting year start day[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]to exist at the appropriate(% style="color:#e74c3c" %)levels(%%)of a data message. In the absence of this[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]], the reporting year start date is assumed to be January 1; therefore if the reporting year coincides with the calendar year, this[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]is not necessary.518 +A specialized attribute (reporting year start day) exists for the purpose of communicating the reporting year start day. This attribute has a fixed identifier (REPORTING_YEAR_START_DAY) and a fixed representation (xs:gMonthDay) so that it can always be easily identified and processed in a data message. Although this attribute exists in specialized sub-class, it functions the same as any other attribute outside of its identification and representation. It must takes its identity from a concept and state its relationship with other components of the data structure definition. The ability to state this relationship allows this reporting year start day attribute to exist at the appropriate levels of a data message. In the absence of this attribute, the reporting year start date is assumed to be January 1; therefore if the reporting year coincides with the calendar year, this Attribute is not necessary. 474 474 475 475 Since the duration and the reporting year start day are known for any reporting period, it is possible to relate any reporting period to a distinct calendar period. The actual Gregorian calendar period covered by the reporting period can be computed as follows (based on the standard format of [REPROTING_YEAR][PERIOD_INDICATOR][PERIOD_VALUE] and the reporting year start day as [REPORTING_YEAR_START_DAY]): 476 476 477 -**~1. Determine [REPORTING_YEAR_BASE]:** 522 +1. **Determine [REPORTING_YEAR_BASE]:** 523 + 478 478 Combine [REPORTING_YEAR] of the reporting period value (YYYY) with [REPORTING_YEAR_START_DAY] (MM-DD) to get a date (YYYY-MM-DD). 525 + 479 479 This is the [REPORTING_YEAR_START_DATE] 480 -**a) If the [PERIOD_INDICATOR] is W: 481 -~1. If [REPORTING_YEAR_START_DATE] is a Friday, Saturday, or Sunday:** 527 + 528 +**a) If the [PERIOD_INDICATOR] is W:** 529 + 530 +1. 531 +11. 532 +111. 533 +1111. **If [REPORTING_YEAR_START_DATE] is a Friday, Saturday, or Sunday:** 534 + 482 482 Add^^3^^ (P3D, P2D, or P1D respectively) to the [REPORTING_YEAR_START_DATE]. The result is the [REPORTING_YEAR_BASE]. 483 483 484 -2. **If [REPORTING_YEAR_START_DATE] is a Monday, Tuesday, Wednesday, or Thursday:** 537 +1. 538 +11. 539 +111. 540 +1111. **If [REPORTING_YEAR_START_DATE] is a Monday, Tuesday, Wednesday, or Thursday:** 541 + 485 485 Add^^3^^ (P0D, -P1D, -P2D, or -P3D respectively) to the [REPORTING_YEAR_START_DATE]. The result is the [REPORTING_YEAR_BASE]. 486 -b) **Else:** 487 -The [REPORTING_YEAR_START_DATE] is the [REPORTING_YEAR_BASE] 488 488 489 -** 2. Determine [PERIOD_DURATION]:**544 +b) **Else:** 490 490 491 -a) If the [PERIOD_INDICATOR] is A, the [PERIOD_DURATION] is P1Y. 492 -b) If the [PERIOD_INDICATOR] is S, the [PERIOD_DURATION] is P6M. 493 -c) If the [PERIOD_INDICATOR] is T, the [PERIOD_DURATION] is P4M. 494 -d) If the [PERIOD_INDICATOR] is Q, the [PERIOD_DURATION] is P3M. 495 -e) If the [PERIOD_INDICATOR] is M, the [PERIOD_DURATION] is P1M. 496 -f) If the [PERIOD_INDICATOR] is W, the [PERIOD_DURATION] is P7D. 497 -g) If the [PERIOD_INDICATOR] is D, the [PERIOD_DURATION] is P1D. 546 +The [REPORTING_YEAR_START_DATE] is the [REPORTING_YEAR_BASE]. 498 498 499 -**3. Determine [PERIOD_START]:** 548 +1. **Determine [PERIOD_DURATION]:** 549 +11. 550 +111. If the [PERIOD_INDICATOR] is A, the [PERIOD_DURATION] is P1Y. 551 +111. If the [PERIOD_INDICATOR] is S, the [PERIOD_DURATION] is P6M. 552 +111. If the [PERIOD_INDICATOR] is T, the [PERIOD_DURATION] is P4M. 553 +111. If the [PERIOD_INDICATOR] is Q, the [PERIOD_DURATION] is P3M. 554 +111. If the [PERIOD_INDICATOR] is M, the [PERIOD_DURATION] is P1M. 555 +111. If the [PERIOD_INDICATOR] is W, the [PERIOD_DURATION] is P7D. 556 +111. If the [PERIOD_INDICATOR] is D, the [PERIOD_DURATION] is P1D. 557 +1. **Determine [PERIOD_START]:** 500 500 501 -Subtract one from the [PERIOD_VALUE] and multiply this by the [PERIOD_DURATION]. Add[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[3~]^^>>path:#_ftn3]](%%) this to the [REPORTING_YEAR_BASE]. The result is the [PERIOD_START].559 +Subtract one from the [PERIOD_VALUE] and multiply this by the [PERIOD_DURATION]. Add[[(% class="wikiinternallink wikiinternallink" %)^^~[3~]^^>>path:#_ftn3]](%%) this to the [REPORTING_YEAR_BASE]. The result is the [PERIOD_START]. 502 502 503 -**4. Determine the [PERIOD_END]:** 561 +1. **Determine the [PERIOD_END]:** 562 + 504 504 Multiply the [PERIOD_VALUE] by the [PERIOD_DURATION]. Add^^3^^ this to the [REPORTING_YEAR_BASE] add^^3^^ -P1D. The result is the [PERIOD_END]. 505 505 506 506 For all of these ranges, the bounds include the beginning of the [PERIOD_START] (i.e. 00:00:00) and the end of the [PERIOD_END] (i.e. 23:59:59). 507 507 508 -**Examples:** 567 +**Examples: ** 509 509 510 510 **2010-Q2, REPORTING_YEAR_START_DAY = ~-~-07-01 (July 1)** 570 + 511 511 ~1. [REPORTING_YEAR_START_DATE] = 2010-07-01 572 + 512 512 b) [REPORTING_YEAR_BASE] = 2010-07-01 513 -[PERIOD_DURATION] = P3M 514 -(2-1) * P3M = P3M 574 + 575 +1. [PERIOD_DURATION] = P3M 576 +1. (2-1) * P3M = P3M 577 + 515 515 2010-07-01 + P3M = 2010-10-01 579 + 516 516 [PERIOD_START] = 2010-10-01 581 + 517 517 4. 2 * P3M = P6M 583 + 518 518 2010-07-01 + P6M = 2010-13-01 = 2011-01-01 585 + 519 519 2011-01-01 + -P1D = 2010-12-31 587 + 520 520 [PERIOD_END] = 2011-12-31 521 521 522 522 The actual calendar range covered by 2010-Q2 (assuming the reporting year begins July 1) is 2010-10-01T00:00:00/2010-12-31T23:59:59 523 523 524 524 **2011-W36, REPORTING_YEAR_START_DAY = ~-~-07-01 (July 1)** 593 + 525 525 ~1. [REPORTING_YEAR_START_DATE] = 2010-07-01 595 + 526 526 a) 2011-07-01 = Friday 597 + 527 527 2011-07-01 + P3D = 2011-07-04 599 + 528 528 [REPORTING_YEAR_BASE] = 2011-07-04 529 -2. [PERIOD_DURATION] = P7D 530 -3. (36-1) * P7D = P245D 601 + 602 +1. [PERIOD_DURATION] = P7D 603 +1. (36-1) * P7D = P245D 604 + 531 531 2011-07-04 + P245D = 2012-03-05 606 + 532 532 [PERIOD_START] = 2012-03-05 608 + 533 533 4. 36 * P7D = P252D 610 + 534 534 2011-07-04 + P252D =2012-03-12 612 + 535 535 2012-03-12 + -P1D = 2012-03-11 614 + 536 536 [PERIOD_END] = 2012-03-11 537 537 538 538 The actual calendar range covered by 2011-W36 (assuming the reporting year begins July 1) is 2012-03-05T00:00:00/2012-03-11T23:59:59 539 539 540 -=== 4.2.7 Distinct Range === 619 +==== 4.2.7 Distinct Range ==== 541 541 542 542 In the case that the reporting period does not fit into one of the prescribe periods above, a distinct time range can be used. The value of these ranges is based on the ISO 8601 time interval format of start/duration. Start can be expressed as either an ISO 8601 date or a date-time, and duration is expressed as an ISO 8601 duration. However, the duration can only be postive. 543 543 544 -=== 4.2.8 Time Format === 623 +==== 4.2.8 Time Format ==== 545 545 546 -In (% style="color:#e74c3c" %)version(%%)2.0 of[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] there is a recommendation to use the[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]to gives additional information on the way time is represented in the message. Following an appraisal of its usefulness this is no longer required. However, it is still possible, if required , to include the[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]].625 +In version 2.0 of SDMX there is a recommendation to use the time format attribute to gives additional information on the way time is represented in the message. Following an appraisal of its usefulness this is no longer required. However, it is still possible, if required , to include the time format attribute in SDMX-ML. 547 547 548 -(% style="width:716.835px" %) 549 -|(% style="width:197px" %)**Code**|(% style="width:517px" %)**Format** 550 -|(% style="width:197px" %)**OTP**|(% style="width:517px" %)Observational [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]]: Superset of all [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[time formats>>doc:sdmx:Glossary.Time format.WebHome]] (Gregorian [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]], Reporting [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]], and Time Range) 551 -|(% style="width:197px" %)**STP**|(% style="width:517px" %)Standard [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]]: Superset of Gregorian and Reporting [[Time Periods>>doc:sdmx:Glossary.Time period.WebHome]] 552 -|(% style="width:197px" %)**GTP**|(% style="width:517px" %)Superset of all Gregorian [[Time Periods>>doc:sdmx:Glossary.Time period.WebHome]] and date-time 553 -|(% style="width:197px" %)**RTP**|(% style="width:517px" %)Superset of all Reporting [[Time Periods>>doc:sdmx:Glossary.Time period.WebHome]] 554 -|(% style="width:197px" %)**TR**|(% style="width:517px" %)Time Range: Start time and duration (YYYY-MMDD(Thh:mm:ss)?/) 555 -|(% style="width:197px" %)**GY**|(% style="width:517px" %)Gregorian Year (YYYY) 556 -|(% style="width:197px" %)**GTM**|(% style="width:517px" %)Gregorian Year Month (YYYY-MM) 557 -|(% style="width:197px" %)**GD**|(% style="width:517px" %)Gregorian Day (YYYY-MM-DD) 558 -|(% style="width:197px" %)**DT**|(% style="width:517px" %)Distinct Point: date-time (YYYY-MM-DDThh:mm:ss) 559 -|(% style="width:197px" %)**RY**|(% style="width:517px" %)Reporting Year (YYYY-A1) 560 -|(% style="width:197px" %)**RS**|(% style="width:517px" %)Reporting Semester (YYYY-Ss) 561 -|(% style="width:197px" %)**RT**|(% style="width:517px" %)Reporting Trimester (YYYY-Tt) 562 -|(% style="width:197px" %)**RQ**|(% style="width:517px" %)Reporting Quarter (YYYY-Qq) 563 -|(% style="width:197px" %)**RM**|(% style="width:517px" %)Reporting Month (YYYY-Mmm) 564 -|(% style="width:197px" %)**[[Code>>doc:sdmx:Glossary.Code.WebHome]]**|(% style="width:517px" %)**Format** 565 -|(% style="width:197px" %)**RW**|(% style="width:517px" %)Reporting Week (YYYY-Www) 566 -|(% style="width:197px" %)**RD**|(% style="width:517px" %)Reporting Day (YYYY-Dddd) 627 +|**Code**|**Format** 628 +|**OTP**|Observational Time Period: Superset of all SDMX time formats (Gregorian Time Period, Reporting Time Period, and Time Range) 629 +|**STP**|Standard Time Period: Superset of Gregorian and Reporting Time Periods 630 +|**GTP**|Superset of all Gregorian Time Periods and date-time 631 +|**RTP**|Superset of all Reporting Time Periods 632 +|**TR**|Time Range: Start time and duration (YYYY-MMDD(Thh:mm:ss)?/<duration>) 633 +|**GY**|Gregorian Year (YYYY) 634 +|**GTM**|Gregorian Year Month (YYYY-MM) 635 +|**GD**|Gregorian Day (YYYY-MM-DD) 636 +|**DT**|Distinct Point: date-time (YYYY-MM-DDThh:mm:ss) 637 +|**RY**|Reporting Year (YYYY-A1) 638 +|**RS**|Reporting Semester (YYYY-Ss) 639 +|**RT**|Reporting Trimester (YYYY-Tt) 640 +|**RQ**|Reporting Quarter (YYYY-Qq) 641 +|**RM**|Reporting Month (YYYY-Mmm) 642 +|**Code**|**Format** 643 +|**RW**|Reporting Week (YYYY-Www) 644 +|**RD**|Reporting Day (YYYY-Dddd) 567 567 568 -**Table 1: SDMX-ML Time Format Codes** 646 + **Table 1: SDMX-ML Time Format Codes** 569 569 570 -=== 4.2.9 Transformation between SDMX-ML and SDMX-EDI === 648 +==== 4.2.9 Transformation between SDMX-ML and SDMX-EDI ==== 571 571 572 -When converting [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]][[data structure definitions>>doc:sdmx:Glossary.Datastructure definition.WebHome]]to [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]][[data structure definitions>>doc:sdmx:Glossary.Data structure definition.WebHome]], only the identifier of the[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]will be retained. The[[representation>>doc:sdmx:Glossary.Representation.WebHome]]of the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]will be converted from the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]format to the fixed[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]][[code list>>doc:sdmx:Glossary.Codelist.WebHome]].If the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]][[data structure definition>>doc:sdmx:Glossary.Datastructuredefinition.WebHome]] does not define a[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]], then one will be automatically created with the identifier "TIME_FORMAT".650 +When converting SDMX-ML data structure definitions to SDMX-EDI data structure definitions, only the identifier of the time format attribute will be retained. The representation of the attribute will be converted from the SDMX-ML format to the fixed SDMX-EDI code list. If the SDMX-ML data structure definition does not define a time format attribute, then one will be automatically created with the identifier "TIME_FORMAT". 573 573 574 -When converting [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]data to[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]], the source[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]will be irrelevant. Since the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]time[[representation>>doc:sdmx:Glossary.Representation.WebHome]]types are not ambiguous, the target[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]]can be determined from the source time value directly. For example, if the[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]time is 2000-Q2 the[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]format will always be 608/708 (depending on whether the target series contains one observation or a range of observations).652 +When converting SDMX-ML data to SDMX-EDI, the source time format attribute will be irrelevant. Since the SDMX-ML time representation types are not ambiguous, the target time format can be determined from the source time value directly. For example, if the SDMX-ML time is 2000-Q2 the SDMX-EDI format will always be 608/708 (depending on whether the target series contains one observation or a range of observations) 575 575 576 -When converting a [[data structure definition>>doc:sdmx:Glossary.Datastructure definition.WebHome]] originating in[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]], the[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]should be ignored, as it serves no purpose in[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]].654 +When converting a data structure definition originating in SDMX-EDI, the time format attribute should be ignored, as it serves no purpose in SDMX-ML. 577 577 578 -When converting data from [[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]to[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]], the source[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]]is only necessary to determine the format of the target time value. For example, a source[[time format>>doc:sdmx:Glossary.Timeformat.WebHome]] of will result in a target time in the format YYYY-Ss whereas a source format of will result in a target time value in the format YYYY-Qq.656 +When converting data from SDMX-EDI to SDMX-ML, the source time format is only necessary to determine the format of the target time value. For example, a source time format of will result in a target time in the format YYYY-Ss whereas a source format of will result in a target time value in the format YYYY-Qq. 579 579 580 -=== 4.2.10 Time Zones === 658 +==== 4.2.10 Time Zones ==== 581 581 582 -In alignment with ISO 8601, [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] allows the specification of a time zone on all[[time periods>>doc:sdmx:Glossary.Timeperiod.WebHome]]and on the reporting year start day. If a time zone is provided on a reporting year start day, then the same time zone (or none) should be reported for each reporting[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]].If the reporting year start day and the reporting period time zone differ, the time zone of the reporting period will take precedence. Examples of each format with time zones are as follows (time zone indicated in bold):660 +In alignment with ISO 8601, SDMX allows the specification of a time zone on all time periods and on the reporting year start day. If a time zone is provided on a reporting year start day, then the same time zone (or none) should be reported for each reporting time period. If the reporting year start day and the reporting period time zone differ, the time zone of the reporting period will take precedence. Examples of each format with time zones are as follows (time zone indicated in bold): 583 583 584 584 * Time Range (start date): 2006-06-05**-05:00**/P5D 585 585 * Time Range (start date-time): 2006-06-05T00:00:00**-05:00**/P5D ... ... @@ -594,56 +594,59 @@ 594 594 * Reporting Month: 2006-M06**-05:00** 595 595 * Reporting Week: 2006-W23**-05:00** 596 596 * Reporting Day: 2006-D156**-05:00** 597 -* Reporting Year Start Day: 07-01**-05:00** 675 +* Reporting Year Start Day: ~-~-07-01**-05:00** 598 598 599 -According to ISO 8601, a date without a time-zone is considered "local time". [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] assumes that local time is that of the sender of the message. In this(% style="color:#e74c3c" %)version(%%)of[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], an optional field is added to the sender definition in the header for specifying a time zone. This field has a default value of 'Z' (UTC). This determination of local time applies for all dates in a message.677 +According to ISO 8601, a date without a time-zone is considered "local time". SDMX assumes that local time is that of the sender of the message. In this version of SDMX, an optional field is added to the sender definition in the header for specifying a time zone. This field has a default value of 'Z' (UTC). This determination of local time applies for all dates in a message. 600 600 601 -=== 4.2.11 Representing Time Spans Elsewhere === 679 +==== 4.2.11 Representing Time Spans Elsewhere ==== 602 602 603 -It has been possible since [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]2.0 for a[[Component>>doc:sdmx:Glossary.Component.WebHome]]to specify a[[representation>>doc:sdmx:Glossary.Representation.WebHome]]of a time span. Depending on the format of the data message, this resulted in either an element with 2 XML[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]for holding the start time and the duration or two separate XML[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]based on the underlying[[Component>>doc:sdmx:Glossary.Component.WebHome]]identifier. For example if REF_PERIOD were given a[[representation>>doc:sdmx:Glossary.Representation.WebHome]]of time span, then in the Compact data format, it would be represented by two XML[[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]; REF_PERIODStartTime (holding the start) and REF_PERIOD (holding the duration). If a new simple type is introduced in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]schemas that can hold ISO 8601 time intervals, then this will no longer be necessary. What was represented as this:681 +It has been possible since SDMX 2.0 for a Component to specify a representation of a time span. Depending on the format of the data message, this resulted in either an element with 2 XML attributes for holding the start time and the duration or two separate XML attributes based on the underlying Component identifier. For example if REF_PERIOD were given a representation of time span, then in the Compact data format, it would be represented by two XML attributes; REF_PERIODStartTime (holding the start) and REF_PERIOD (holding the duration). If a new simple type is introduced in the SDMX schemas that can hold ISO 8601 time intervals, then this will no longer be necessary. What was represented as this: 604 604 605 -<Series REF_PERIODStartTime="2000-01-01T00:00:00" REF_PERIOD="P2M"/> 683 + <Series REF_PERIODStartTime="2000-01-01T00:00:00" REF_PERIOD="P2M"/> 606 606 607 607 can now be represented with this: 608 608 609 609 <Series REF_PERIOD="2000-01-01T00:00:00/P2M"/> 610 610 611 -=== 4.2.12 Notes on Formats === 689 +==== 4.2.12 Notes on Formats ==== 612 612 613 -There is no ambiguity in these formats so that for any given value of time, the [[category>>doc:sdmx:Glossary.Category.WebHome]]of the period (and thus the intended[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]] range) is always clear. It should also be noted that by utilizing the ISO 8601 format, and a format loosely based on it for the report periods, the values of time can easily be sorted chronologically without additional parsing.691 +There is no ambiguity in these formats so that for any given value of time, the category of the period (and thus the intended time period range) is always clear. It should also be noted that by utilizing the ISO 8601 format, and a format loosely based on it for the report periods, the values of time can easily be sorted chronologically without additional parsing. 614 614 615 -=== 4.2.13 Effect on Time Ranges === 693 +==== 4.2.13 Effect on Time Ranges ==== 616 616 617 -All [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]data messages are capable of functioning in a manner similar to[[SDMX-EDI>>doc:sdmx:Glossary.SDMX-EDI.WebHome]]if the[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]at the observation(% style="color:#e74c3c" %)level(%%)is time: the[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]]for the first observation can be stated and the rest of the observations can omit the time value as it can be derived from the start time and the frequency. Since the frequency can be determined based on the actual format of the time value for everything but distinct points in time and time ranges, this makes is even simpler to process as the interval between time ranges is known directly from the time value.695 +All SDMX-ML data messages are capable of functioning in a manner similar to SDMX-EDI if the Dimension at the observation level is time: the time period for the first observation can be stated and the rest of the observations can omit the time value as it can be derived from the start time and the frequency. Since the frequency can be determined based on the actual format of the time value for everything but distinct points in time and time ranges, this makes is even simpler to process as the interval between time ranges is known directly from the time value. 618 618 619 -=== 4.2.14 Time in Query Messages === 697 +==== 4.2.14 Time in Query Messages ==== 620 620 621 -When querying for time values, the value of a time parameter can be provided as any of the Observational [[Time Period>>doc:sdmx:Glossary.Timeperiod.WebHome]]formats and must be paired with an operator. In addition, an explicit value for the reporting year start day can be provided, or this can be set to "Any". This section will detail how systems processing query messages should interpret these parameters.699 +When querying for time values, the value of a time parameter can be provided as any of the Observational Time Period formats and must be paired with an operator. In addition, an explicit value for the reporting year start day can be provided, or this can be set to "Any". This section will detail how systems processing query messages should interpret these parameters. 622 622 623 -Fundamental to processing a time value parameter in a query message is understanding that all [[time periods>>doc:sdmx:Glossary.Timeperiod.WebHome]]should be handled as a distinct range of time. Since the time parameter in the query is paired with an operator, this is also effectively represents a distinct range of time. Therefore, a system processing the query must simply match the data where the[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]]for requested parameter is encompassed by the[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]] resulting from value of the query parameter. The following table details how the operators should be interpreted for any[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]] provided as a parameter.701 +Fundamental to processing a time value parameter in a query message is understanding that all time periods should be handled as a distinct range of time. Since the time parameter in the query is paired with an operator, this is also effectively represents a distinct range of time. Therefore, a system processing the query must simply match the data where the time period for requested parameter is encompassed by the time period resulting from value of the query parameter. The following table details how the operators should be interpreted for any time period provided as a parameter. 624 624 625 -(% style="width:1024.29px" %) 626 -|(% style="width:238px" %)**Operator**|(% style="width:782px" %)**Rule** 627 -|(% style="width:238px" %)Greater Than|(% style="width:782px" %)Any data after the last moment of the period 628 -|(% style="width:238px" %)Less Than|(% style="width:782px" %)Any data before the first moment of the period 629 -|(% style="width:238px" %)Greater Than or Equal To|(% style="width:782px" %)((( 630 -Any data on or after the first moment of the period 703 +|**Operator**|**Rule** 704 +|Greater Than|Any data after the last moment of the period 705 +|Less Than|Any data before the first moment of the period 706 +|Greater Than or Equal To|((( 707 +Any data on or after the first moment of 708 + 709 +the period 631 631 ))) 632 -| (% style="width:238px" %)Less Than or Equal To|(% style="width:782px" %)Any data on or before the last moment of the period633 -| (% style="width:238px" %)Equal To|(% style="width:782px" %)Any data which falls on or after the first moment of the period and before or on the last moment of the period711 +|Less Than or Equal To|Any data on or before the last moment of the period 712 +|Equal To|Any data which falls on or after the first moment of the period and before or on the last moment of the period 634 634 635 -Reporting [[Time Periods>>doc:sdmx:Glossary.Timeperiod.WebHome]]as query parameters are handled based on whether the value of the reportingYearStartDay XML[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]is an explicit month and day or "Any":714 +Reporting Time Periods as query parameters are handled based on whether the value of the reportingYearStartDay XML attribute is an explicit month and day or "Any": 636 636 637 -If the time parameter provides an explicit month and day value for the reportingYearStartDay XML [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]], then the parameter value is converted to a distinct range and processed as any other[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]]would be processed.716 +If the time parameter provides an explicit month and day value for the reportingYearStartDay XML attribute, then the parameter value is converted to a distinct range and processed as any other time period would be processed. 638 638 639 -If the reportingYeartStartDay XML [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]has a value of "Any", then any data within the bounds of the reporting period for the year is matched, regardless of the actual start day of the reporting year. In addition, data reported against a normal calendar period is matched if it falls within the bounds of the time parameter based on a reporting year start day of January 1. When determining whether another reporting period falls within the bounds of a report period query parameter, one will have to take into account the actual[[time period>>doc:sdmx:Glossary.Timeperiod.WebHome]]to compare weeks and days to higher order report periods. This will be demonstrated in the examples to follow.718 +If the reportingYeartStartDay XML attribute has a value of "Any", then any data within the bounds of the reporting period for the year is matched, regardless of the actual start day of the reporting year. In addition, data reported against a normal calendar period is matched if it falls within the bounds of the time parameter based on a reporting year start day of January 1. When determining whether another reporting period falls within the bounds of a report period query parameter, one will have to take into account the actual time period to compare weeks and days to higher order report periods. This will be demonstrated in the examples to follow. 640 640 641 -Note that the reportingYearStartDay XML [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]on the time value parameter is only used to qualify a reporting period value for the given time value parameter. The usage of this is different than using the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value parameter for the actual reporting year start day[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]].In the case that the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value parameters is used for the reporting year start day data structure[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]], it will be treated as any other[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value parameter; data will be filtered to that which matches the values specified for the given[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]].For example, if the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value parameter references the reporting year start day[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]and specifies a value of "07-01", then only data which has this[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]with the value "07-01" will be returned. In terms of processing any time value parameters, the value supplied in the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value parameter will be irrelevant.720 +Note that the reportingYearStartDay XML attribute on the time value parameter is only used to qualify a reporting period value for the given time value parameter. The usage of this is different than using the attribute value parameter for the actual reporting year start day attribute. In the case that the attribute value parameters is used for the reporting year start day data structure attribute, it will be treated as any other attribute value parameter; data will be filtered to that which matches the values specified for the given attribute. For example, if the attribute value parameter references the reporting year start day attribute and specifies a value of "~-~-07-01", then only data which has this attribute with the value "~-~-07-01" will be returned. In terms of processing any time value parameters, the value supplied in the attribute value parameter will be irrelevant. 642 642 643 643 **Examples:** 644 644 645 645 **Gregorian Period** 725 + 646 646 Query Parameter: Greater than 2010 727 + 647 647 Literal Interpretation: Any data where the start period occurs after 2010-1231T23:59:59. 648 648 649 649 Example Matches: ... ... @@ -663,10 +663,13 @@ 663 663 **Reporting Period with explicit start day** 664 664 665 665 Query Parameter: Greater than or equal to 2009-Q3, reporting year start day = "-07-01" 747 + 666 666 Literal Interpretation: Any data where the start period occurs on after 2010-0101T00:00:00 (Note that in this case 2009-Q3 is converted to the explicit date range of 2010-01-01/2010-03-31 because of the reporting year start day value). Example Matches: Same as previous example 667 667 668 668 **Reporting Period with "Any" start day** 751 + 669 669 Query Parameter: Greater than or equal to 2010-Q3, reporting year start day = "Any" 753 + 670 670 Literal Interpretation: Any data with a reporting period where the start period is on or after the start period of 2010-Q3 for the same reporting year start day, or and data where the start period is on or after 2010-07-01. Example Matches: 671 671 672 672 * 2011 or later ... ... @@ -678,57 +678,62 @@ 678 678 * 2010-T3 (any reporting year start day) 679 679 * 2010-Q3 or later (any reporting year start day) 680 680 * 2010-M07 or later (any reporting year start day) 681 -* 2010-W27 or later (reporting year start day ~-~-01-01){{footnote}}2010-Q3 (with a reporting year start day of --01-01) starts on 2010-07-01. This is day 4 of week 26, therefore the first week matched is week 27.{{/footnote}} 2010-D182 or later (reporting year start day ~-~-01-01) 682 -* 2010-W28 or later (reporting year start day ~-~-07-01){{footnote}}2010-Q3 (with a reporting year start day of --07-01) starts on 2011-01-01. This is day 6 of week 27, therefore the first week matched is week 28.{{/footnote}} 683 -* 2010-D185 or later (reporting year start day ~-~-07-01) 765 +* 2010-W27 or later (reporting year start day ~-~-01-01)^^4^^ 2010-D182 or later (reporting year start day ~-~-01-01) 766 +* 2010-W28 or later (reporting year start day ~-~-07-01)^^5^^ 684 684 768 +^^4^^ 2010-Q3 (with a reporting year start day of ~-~-01-01) starts on 2010-07-01. This is day 4 of week 26, therefore the first week matched is week 27. 769 + 770 + 2010-D185 or later (reporting year start day ~-~-07-01) 771 + 685 685 == 4.3 Structural Metadata Querying Best Practices == 686 686 687 -When querying for [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]], the ability to state how references should be resolved is quite powerful. However, this mechanism is not always necessary and can create an undue burden on the systems processing the queries if it is not used properly.774 +When querying for structural metadata, the ability to state how references should be resolved is quite powerful. However, this mechanism is not always necessary and can create an undue burden on the systems processing the queries if it is not used properly. 688 688 689 -Any [[structural metadata>>doc:sdmx:Glossary.Structuralmetadata.WebHome]] object which contains a reference to an object can be queried based on that reference. For example, a categorisation references both a[[category>>doc:sdmx:Glossary.Category.WebHome]]and the object is it categorising. As this is the case, one can query for categorisations which categorise a particular object or which categorise against a particular[[category>>doc:sdmx:Glossary.Category.WebHome]]or[[category scheme>>doc:sdmx:Glossary.Categoryscheme.WebHome]].This mechanism should be used when the referenced object is known.776 +Any structural metadata object which contains a reference to an object can be queried based on that reference. For example, a categorisation references both a category and the object is it categorising. As this is the case, one can query for categorisations which categorise a particular object or which categorise against a particular category or category scheme. This mechanism should be used when the referenced object is known. 690 690 691 -When the referenced object is not known, then the reference resolution mechanism could be used. For example, suppose one wanted to find all [[category schemes>>doc:sdmx:Glossary.Categoryscheme.WebHome]]and the related categorisations for a given[[maintenance agency>>doc:sdmx:Glossary.Maintenanceagency.WebHome]].In this case, one could query for the[[category scheme>>doc:sdmx:Glossary.Categoryscheme.WebHome]] by the[[maintenance agency>>doc:sdmx:Glossary.Maintenanceagency.WebHome]] and specify that parent and sibling references should be resolved. This would result in the categorisations which reference the[[categories>>doc:sdmx:Glossary.Category.WebHome]]in the matched schemes to be returned, as well as the object which they categorise.778 +When the referenced object is not known, then the reference resolution mechanism could be used. For example, suppose one wanted to find all category schemes and the related categorisations for a given maintenance agency. In this case, one could query for the category scheme by the maintenance agency and specify that parent and sibling references should be resolved. This would result in the categorisations which reference the categories in the matched schemes to be returned, as well as the object which they categorise. 692 692 693 693 == 4.4 Versioning and External Referencing == 694 694 695 -Within the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]Structure Message, there is a pattern for versioning and external referencing which should be pointed out. The identifiers are qualified by their(% style="color:#e74c3c" %)version(%%)numbers – that is, an object with an Agency of “A”, and ID of “X” and a(% style="color:#e74c3c" %)version(%%)of “1.0” is a different object than one with an Agency of “A’, an ID of “X”, and a(% style="color:#e74c3c" %)version(%%)of “1.1”.782 +Within the SDMX-ML Structure Message, there is a pattern for versioning and external referencing which should be pointed out. The identifiers are qualified by their version numbers – that is, an object with an Agency of “A”, and ID of “X” and a version of “1.0” is a different object than one with an Agency of “A’, an ID of “X”, and a version of “1.1”. 696 696 697 -The production (% style="color:#e74c3c" %)versions(%%)of identifiable objects/resources are assumed to be static – that is, they have their isFinal[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]set to ‘true”. Once in production, and object cannot change in any way, or it must be versioned. For cases where an object is not static, the isFinal[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]must have a value of “false”, but non-final objects should not be used outside of a specific system designed to accommodate them. For most purposes, all objects should be declared final before use in production.784 +The production versions of identifiable objects/resources are assumed to be static – that is, they have their isFinal attribute set to ‘true”. Once in production, and object cannot change in any way, or it must be versioned. For cases where an object is not static, the isFinal attribute must have a value of “false”, but non-final objects should not be used outside of a specific system designed to accommodate them. For most purposes, all objects should be declared final before use in production. 698 698 699 -This mechanism is an “early binding” one – everything with a versioned identity is a known quantity, and will not change. It is worth pointing out that in some cases relationships are essentially one-way references: an illustrative case is that of [[Categories>>doc:sdmx:Glossary.Category.WebHome]].While a[[Category>>doc:sdmx:Glossary.Category.WebHome]]may be referenced by many[[dataflows>>doc:sdmx:Glossary.Dataflow.WebHome]]and[[metadata flows>>doc:sdmx:Glossary.Metadataflow.WebHome]], the addition of more references from flow objects does not(% style="color:#e74c3c" %)version(%%)the[[Category>>doc:sdmx:Glossary.Category.WebHome]].This is because the flows are not properties of the[[Categories>>doc:sdmx:Glossary.Category.WebHome]]– they merely make references to it. If the name of a[[Category>>doc:sdmx:Glossary.Category.WebHome]]changed, or its subCategories changed, then versioning would be necessary.786 +This mechanism is an “early binding” one – everything with a versioned identity is a known quantity, and will not change. It is worth pointing out that in some cases relationships are essentially one-way references: an illustrative case is that of Categories. While a Category may be referenced by many dataflows and metadata flows, the addition of more references from flow objects does not version the Category. This is because the flows are not properties of the Categories – they merely make references to it. If the name of a Category changed, or its subCategories changed, then versioning would be necessary. 700 700 701 - Versioningoperatesatthe(%style="color:#e74c3c" %)level(%%)of versionableandmaintainableobjectsinthe [[SDMX information model>>doc:sdmx:Glossary.SDMXInformationModel.WebHome]].Ifany ofthechildren ofobjectsatthese(%style="color:#e74c3c"%)levels(%%)change,thenthe objectsthemselvesare versioned.788 +^^5^^ 2010-Q3 (with a reporting year start day of ~-~-07-01) starts on 2011-01-01. This is day 6 of week 27, therefore the first week matched is week 28. 702 702 703 - Onearea which ismuchimpacted by this versioningscheme is the ability toreference external objects. With themany dependencieswithinthevarious structuralobjects in [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]], it is usefulto havea scheme forexternal referencing. Thisis doneat the (% style="color:#e74c3c"%)level(%%)ofmaintainable objects(DSDs, [[code lists>>doc:sdmx:Glossary.Code list.WebHome]], [[conceptschemes>>doc:sdmx:Glossary.Conceptscheme.WebHome]],etc.) In an [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]Structure Message, whenever an “[[isExternalReference>>doc:sdmx:Glossary.isExternalReference.WebHome]]” [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]issetto true, thenthe application must resolve the address provided inthe associated“uri” [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]and usethe[[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]StructureMessagestored at that locationforthefull definitionofthe objectin question.Alternately, if a registry“urn” [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]has been provided, theregistry can be used to supply the full detailsof theobject.790 +Versioning operates at the level of versionable and maintainable objects in the SDMX information model. If any of the children of objects at these levels change, then the objects themselves are versioned. 704 704 705 - Becausethe(%style="color:#e74c3c"%)version(%%)numberispart of the identifierfor anobject,(%style="color:#e74c3c"%)versions(%%)are anecessarypartof determiningthatagivenresource istheonewhichwascalled for. Itshouldbenotedthatwhenever a(%style="color:#e74c3c" %)version(%%) numberisnotsupplied,itis assumed tobe “1.0”.(The“x.x”versioningnotationis conventionalinpracticewith [[SDMX>>doc:sdmx:Glossary.Statisticaldataandmetadataexchange.WebHome]],butnotrequired.)792 +One area which is much impacted by this versioning scheme is the ability to reference external objects. With the many dependencies within the various structural objects in SDMX, it is useful to have a scheme for external referencing. This is done at the level of maintainable objects (DSDs, code lists, concept schemes, etc.) In an SDMX-ML Structure Message, whenever an “isExternalReference” attribute is set to true, then the application must resolve the address provided in the associated “uri” attribute and use the SDMX-ML Structure Message stored at that location for the full definition of the object in question. Alternately, if a registry “urn” attribute has been provided, the registry can be used to supply the full details of the object. 706 706 794 +Because the version number is part of the identifier for an object, versions are a necessary part of determining that a given resource is the one which was called for. It should be noted that whenever a version number is not supplied, it is assumed to be “1.0”. (The “x.x” versioning notation is conventional in practice with SDMX, but not required.) 795 + 707 707 = 5 Metadata Structure Definition (MSD) = 708 708 709 709 == 5.1 Scope == 710 710 711 -The scope of the [[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] is enhanced in this(% style="color:#e74c3c" %)version(%%)to better support the types of construct to which metadata can be attached. In particular it is possible to specify an attachment to any key or partial key of a[[data set>>doc:sdmx:Glossary.Dataset.WebHome]].This is particularly useful for web dissemination where metadata may be present for the data, but is not stored with the data but is related to it. For this use case to be supported it is necessary to be able to specify in the[[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] that metadata is attached to a key or partial key, and the actual key or partial key to be identified in the[[Metadata Set>>doc:sdmx:Glossary.Metadata set.WebHome]].800 +The scope of the MSD is enhanced in this version to better support the types of construct to which metadata can be attached. In particular it is possible to specify an attachment to any key or partial key of a data set. This is particularly useful for web dissemination where metadata may be present for the data, but is not stored with the data but is related to it. For this use case to be supported it is necessary to be able to specify in the MSD that metadata is attached to a key or partial key, and the actual key or partial key to be identified in the Metadata Set. 712 712 713 -In addition to the increase in the scope of objects that can be included in an [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], the way the identifier mechanism works in this(% style="color:#e74c3c" %)version(%%), and the terminology used, is much simpler.802 +In addition to the increase in the scope of objects that can be included in an MSD, the way the identifier mechanism works in this version, and the terminology used, is much simpler. 714 714 715 715 == 5.2 Identification of the Object Type to which the Metadata is to be Attached == 716 716 717 -The following example shows the structure and naming of the [[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] [[components>>doc:sdmx:Glossary.Component.WebHome]]for the use case of defining full and partial keys.806 +The following example shows the structure and naming of the MSD components for the use case of defining full and partial keys. 718 718 719 -The schematic structure of an [[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] is shown below.808 +The schematic structure of an MSD is shown below. 720 720 721 721 [[image:1747836776649-282.jpeg]] 722 722 723 -** Figure1: Schematic of the Metadata Structure Definition**812 +1. **1: Schematic of the Metadata Structure Definition** 724 724 725 -The [[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] comprises the specification of the object types to which metadata can be reported in a[[Metadata Set>>doc:sdmx:Glossary.Metadataset.WebHome]](Metadata Target(s)), and the Report Structure(s) comprising the Metadata[[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]]that identify the(% style="color:#e74c3c" %)Concept(%%)for which metadata may be reported in the[[Metadata Set>>doc:sdmx:Glossary.Metadataset.WebHome]].Importantly, one Report Structure references the Metadata Target for which it is relevant. One Report Structure can reference many Metadata Target i.e. the same Report Structure can be used for different target objects.814 +The MSD comprises the specification of the object types to which metadata can be reported in a Metadata Set (Metadata Target(s)), and the Report Structure(s) comprising the Metadata Attributes that identify the Concept for which metadata may be reported in the Metadata Set. Importantly, one Report Structure references the Metadata Target for which it is relevant. One Report Structure can reference many Metadata Target i.e. the same Report Structure can be used for different target objects. 726 726 727 727 [[image:1747836776655-364.jpeg]] 728 728 729 -** Figure2: Example[[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] showing Metadata Targets**818 +1. **2: Example MSD showing Metadata Targets** 730 730 731 -Note that the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]schemas have explicit XML elements for each identifiable object type because identifying, for instance, a Maintainable Object has different properties from an Identifiable Object which must also include the agencyId,(% style="color:#e74c3c" %)version(%%), and id of the Maintainable Object in which it resides.820 +Note that the SDMX-ML schemas have explicit XML elements for each identifiable object type because identifying, for instance, a Maintainable Object has different properties from an Identifiable Object which must also include the agencyId, version, and id of the Maintainable Object in which it resides. 732 732 733 733 == 5.3 Report Structure == 734 734 ... ... @@ -736,37 +736,38 @@ 736 736 737 737 [[image:1747836776658-510.jpeg]] 738 738 739 -**Figure 3: Example MSD showing specification of three Metadata Attributes** 828 +**Figure 3: Example MSD showing specification of three Metadata Attributes **This example shows the following hierarchy of Metadata Attributes: 740 740 741 - Thisexampleshowsthe following[[hierarchy>>doc:sdmx:Glossary.Hierarchy.WebHome]]ofMetadata[[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]]:830 +Source – this is presentational and no metadata is expected to be reported at this level 742 742 743 -Source – this is presentational and no metadata is expected to be reported at this (% style="color:#e74c3c" %)level 744 - 745 745 * Source Type 746 746 * Collection Source Name 747 747 748 748 == 5.4 Metadata Set == 749 749 750 -An example of reporting metadata according to the [[MSD>>doc:sdmx:Glossary.Metadatastructuredefinition.WebHome]] described above, is shown below.837 +An example of reporting metadata according to the MSD described above, is shown below. 751 751 752 752 [[image:1747836776677-246.jpeg]] 753 753 754 -**Figure 4: Example Metadata Set **This example shows: 841 + **Figure 4: Example Metadata Set **This example shows: 755 755 756 -1. The reference to the [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], Metadata Report, and Metadata Target (MetadataTargetValue) 757 -1. The reported metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] (AttributeValueSet) 843 +1. The reference to the MSD, Metadata Report, and Metadata Target 758 758 845 +(MetadataTargetValue) 846 + 847 +1. The reported metadata attributes (AttributeValueSet) 848 + 759 759 = 6 Maintenance Agencies = 760 760 761 -All [[structural metadata>>doc:sdmx:Glossary.Structuralmetadata.WebHome]]in[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]is owned and maintained by a[[maintenance agency>>doc:sdmx:Glossary.Maintenanceagency.WebHome]](Agency identified by agencyID in the schemas). It is vital to the integrity of the[[structural metadata>>doc:sdmx:Glossary.Structuralmetadata.WebHome]] that there are no conflicts in agencyID. In order to achieve this[[SDMX>>doc:sdmx:Glossary.Statisticaldata andmetadata exchange.WebHome]] adopts the following rules:851 +All structural metadata in SDMX is owned and maintained by a maintenance agency (Agency identified by agencyID in the schemas). It is vital to the integrity of the structural metadata that there are no conflicts in agencyID. In order to achieve this SDMX adopts the following rules: 762 762 763 -1. Agencies are maintained in an [[Agency Scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]](which is a sub class of Organisation Scheme)764 -1. The [[maintenance agency>>doc:sdmx:Glossary.Maintenanceagency.WebHome]] of the[[Agency Scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]] must also be declared in a (different)[[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]].765 -1. The “top- (% style="color:#e74c3c" %)level(%%)” agency is[[SDMX>>doc:sdmx:Glossary.Statisticaldata andmetadata exchange.WebHome]] and this[[agency scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]]is maintained by[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].766 -1. Agencies registered in the top- (% style="color:#e74c3c" %)level(%%)scheme can themselves maintain a single[[Agency Scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]]. [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]is an agency in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[agency scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]].Agencies in this scheme can themselves maintain a single[[Agency Scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]]and so on.767 -1. The AgencyScheme cannot be versioned and so take a default (% style="color:#e74c3c" %)version(%%)number of 1.0 and cannot be made “final”.853 +1. Agencies are maintained in an Agency Scheme (which is a sub class of Organisation Scheme) 854 +1. The maintenance agency of the Agency Scheme must also be declared in a (different) Agency Scheme. 855 +1. The “top-level” agency is SDMX and this agency scheme is maintained by SDMX. 856 +1. Agencies registered in the top-level scheme can themselves maintain a single Agency Scheme. SDMX is an agency in the SDMX agency scheme. Agencies in this scheme can themselves maintain a single Agency Scheme and so on. 857 +1. The AgencyScheme cannot be versioned and so take a default version number of 1.0 and cannot be made “final”. 768 768 1. There can be only one AgencyScheme maintained by any one Agency. It has a fixed Id of AgencyScheme. 769 -1. The format of the agency identifier is agencyId.agencyID etc. The top- (% style="color:#e74c3c" %)level(%%)agency in this identification mechanism is the agency registered in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[agency scheme>>doc:sdmx:Glossary.Agencyscheme.WebHome]].In other words,[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]is not a part of the hierarchical ID structure for agencies.[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]is, itself, a[[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]].859 +1. The format of the agency identifier is agencyId.agencyID etc. The top-level agency in this identification mechanism is the agency registered in the SDMX agency scheme. In other words, SDMX is not a part of the hierarchical ID structure for agencies. SDMX is, itself, a maintenance agency. 770 770 771 771 This supports a hierarchical structure of agencyID. 772 772 ... ... @@ -774,9 +774,9 @@ 774 774 775 775 [[image:1747836776680-229.jpeg]] 776 776 777 -**Figure 5: Example of Hierarchic Structure of Agencies** 867 + **Figure 5: Example of Hierarchic Structure of Agencies** 778 778 779 -Each agency is identified by its full [[hierarchy>>doc:sdmx:Glossary.Hierarchy.WebHome]]excluding[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].869 +Each agency is identified by its full hierarchy excluding SDMX. 780 780 781 781 The XML representing this structure is shown below. 782 782 ... ... @@ -790,271 +790,342 @@ 790 790 791 791 **Figure 7: Example Showing Use of Agency Identifiers** 792 792 793 -Each of these [[maintenance agencies>>doc:sdmx:Glossary.Maintenanceagency.WebHome]]has an identical Codelist with the Id CL_BOP. However, each is uniquely identified by means of the hierarchic agency structure.883 +Each of these maintenance agencies has an identical Codelist with the Id CL_BOP. However, each is uniquely identified by means of the hierarchic agency structure. 794 794 795 795 = 7 Concept Roles = 796 796 797 797 == 7.1 Overview == 798 798 799 -The [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] [[Components>>doc:sdmx:Glossary.Component.WebHome]]of[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]and[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]can play a specific role in the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and it is important to some applications that this role is specified. For instance, the following roles are some examples:889 +The DSD Components of Dimension and Attribute can play a specific role in the DSD and it is important to some applications that this role is specified. For instance, the following roles are some examples: 800 800 801 -**Frequency **– in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] the content of this [[Component>>doc:sdmx:Glossary.Component.WebHome]] contains information on the frequency of the observation values 802 -**Geography** - in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] the content of this [[Component>>doc:sdmx:Glossary.Component.WebHome]] contains information on the geographic location of the observation values 803 -**Unit** **of [[Measure>>doc:sdmx:Glossary.Measure.WebHome]]** - in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] the content of this [[Component>>doc:sdmx:Glossary.Component.WebHome]] contains information on the [[unit of measure>>doc:sdmx:Glossary.Unit of measure.WebHome]] of the observation values 891 +**Frequency **– in a data set the content of this Component contains information on the frequency of the observation values 804 804 805 - In order fortheserolesto be extensible andalsoto enableuser communitiesto maintain community-specific roles, theroles are maintained ina controlledvocabularywhich isimplemented in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] as (% style="color:#e74c3c"%)Concepts(%%) in a [[ConceptScheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]].The [[Component>>doc:sdmx:Glossary.Component.WebHome]] optionallyreferencesthis (% style="color:#e74c3c"%)Concept(%%) if it is required todeclare the role explicitly.Note that a [[Component>>doc:sdmx:Glossary.Component.WebHome]] canplay morethan oneroleandtherefore multiple “role” (% style="color:#e74c3c" %)concepts(%%)can bereferenced.893 +**Geography** - in a data set the content of this Component contains information on the geographic location of the observation values 806 806 895 +**Unit** **of Measure** - in a data set the content of this Component contains information on the unit of measure of the observation values 896 + 897 +In order for these roles to be extensible and also to enable user communities to maintain community-specific roles, the roles are maintained in a controlled vocabulary which is implemented in SDMX as Concepts in a Concept Scheme. The Component optionally references this Concept if it is required to declare the role explicitly. Note that a Component can play more than one role and therefore multiple “role” concepts can be referenced. 898 + 807 807 == 7.2 Information Model == 808 808 809 809 The Information Model for this is shown below: 810 810 811 -[[image:1747855024745-946.png]] 812 812 813 -**Figure 8: Information Model Extract for Concept Role** 904 + **Figure 8: Information Model Extract for Concept Role** 814 814 815 -It is possible to specify zero or more (% style="color:#e74c3c" %)concept(%%)roles for a[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]and Data[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]](but not the ReportingYearStartDay). The Time[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], Primary[[Measure>>doc:sdmx:Glossary.Measure.WebHome]], and the[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]ReportingYearStartDay have explicitly defined roles and cannot be further specified with additional(% style="color:#e74c3c" %)concept(%%)roles.906 +It is possible to specify zero or more concept roles for a Dimension, Measure Dimension and Data Attribute (but not the ReportingYearStartDay). The Time Dimension, Primary Measure, and the Attribute ReportingYearStartDay have explicitly defined roles and cannot be further specified with additional concept roles. 816 816 817 817 == 7.3 Technical Mechanism == 818 818 819 -The mechanism for maintain and using (% style="color:#e74c3c" %)concept(%%)roles is as follows:910 +The mechanism for maintain and using concept roles is as follows: 820 820 821 -1. Any recognized Agency can have a [[concept scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]]that contains(% style="color:#e74c3c" %)concepts(%%)that identify(% style="color:#e74c3c" %)concept(%%)roles. Indeed, from a technical perspective any agency can have more than one of these schemes, though this is not recommended.822 -1. The [[concept scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]]that contains the “role”(% style="color:#e74c3c" %)concepts(%%)can contain(% style="color:#e74c3c" %)concepts(%%)that do not play a role.823 -1. There is no explicit indication on the (% style="color:#e74c3c" %)Concept(%%)whether it is a ‘role”(% style="color:#e74c3c" %)concept(%%).824 -1. Therefore, any (% style="color:#e74c3c" %)concept(%%)in any[[concept scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]]is capable of being a “role”(% style="color:#e74c3c" %)concept(%%).825 -1. It is the responsibility of Agencies to ensure their community knows which (% style="color:#e74c3c" %)concepts(%%)in which[[concept schemes>>doc:sdmx:Glossary.Conceptscheme.WebHome]]play a “role” and the significance and interpretation of this role. In other words, such(% style="color:#e74c3c" %)concepts(%%)must be known by applications, there is no technical mechanism that can inform an application on how to process such a “role”.826 -1. If the (% style="color:#e74c3c" %)concept(%%)referenced in the(% style="color:#e74c3c" %)Concept(%%)Identity in a[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] [[component>>doc:sdmx:Glossary.Component.WebHome]]([[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]) is contained in the[[concept scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]] containing(% style="color:#e74c3c" %)concept(%%)roles then the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] [[component>>doc:sdmx:Glossary.Component.WebHome]]could play the role implied by the(% style="color:#e74c3c" %)concept(%%), if this is understood by the processing application.827 -1. If the (% style="color:#e74c3c" %)concept(%%)referenced in the(% style="color:#e74c3c" %)Concept(%%)Identity in a[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] [[component>>doc:sdmx:Glossary.Component.WebHome]]([[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]) is not contained in the[[concept scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]] containing(% style="color:#e74c3c" %)concept(%%)roles, and the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] [[component>>doc:sdmx:Glossary.Component.WebHome]]is playing a role, then the(% style="color:#e74c3c" %)concept(%%)role is identified by the(% style="color:#e74c3c" %)Concept(%%)Role in the schema.912 +1. Any recognized Agency can have a concept scheme that contains concepts that identify concept roles. Indeed, from a technical perspective any agency can have more than one of these schemes, though this is not recommended. 913 +1. The concept scheme that contains the “role” concepts can contain concepts that do not play a role. 914 +1. There is no explicit indication on the Concept whether it is a ‘role” concept. 915 +1. Therefore, any concept in any concept scheme is capable of being a “role” concept. 916 +1. It is the responsibility of Agencies to ensure their community knows which concepts in which concept schemes play a “role” and the significance and interpretation of this role. In other words, such concepts must be known by applications, there is no technical mechanism that can inform an application on how to process such a “role”. 917 +1. If the concept referenced in the Concept Identity in a DSD component (Dimension, Measure Dimension, Attribute) is contained in the concept scheme containing concept roles then the DSD component could play the role implied by the concept, if this is understood by the processing application. 918 +1. If the concept referenced in the Concept Identity in a DSD component (Dimension, Measure Dimension, Attribute) is not contained in the concept scheme containing concept roles, and the DSD component is playing a role, then the concept role is identified by the Concept Role in the schema. 828 828 829 829 == 7.4 SDMX-ML Examples in a DSD == 830 830 831 -The [[Cross-Domain Concept>>doc:sdmx:Glossary.Cross-domainconcept.WebHome]]Scheme maintained by[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] contains(% style="color:#e74c3c" %)concept(%%)role(% style="color:#e74c3c" %)concepts(%%)(FREQ chosen as an example).922 +The Cross-Domain Concept Scheme maintained by SDMX contains concept role concepts (FREQ chosen as an example). 832 832 833 -[[image:17478 55054559-410.png]]924 +[[image:1747836776691-440.jpeg]] 834 834 835 -Whether this is a role or not depends upon the application understanding that FREQ in the [[Cross-Domain Concept>>doc:sdmx:Glossary.Cross-domainconcept.WebHome]]Scheme is a role of Frequency.926 +Whether this is a role or not depends upon the application understanding that FREQ in the Cross-Domain Concept Scheme is a role of Frequency. 836 836 837 -Using a [[Concept Scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]]that is not the[[Cross-Domain Concept>>doc:sdmx:Glossary.Cross-domainconcept.WebHome]]Scheme where it is required to assign a role using the[[Cross-Domain Concept>>doc:sdmx:Glossary.Cross-domainconcept.WebHome]]Scheme. Again FREQ is chosen as the example.928 +Using a Concept Scheme that is not the Cross-Domain Concept Scheme where it is required to assign a role using the Cross-Domain Concept Scheme. Again FREQ is chosen as the example. 838 838 839 -[[image:17478 55075263-887.png]]930 +[[image:1747836776693-898.jpeg]] 840 840 841 -This explicitly states that this [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is playing a role identified by the FREQ (% style="color:#e74c3c" %)concept(%%) in the [[Cross-Domain Concept>>doc:sdmx:Glossary.Cross-domain concept.WebHome]] Scheme. Again the application needs to understand what FREQ in the [[Cross-Domain Concept>>doc:sdmx:Glossary.Cross-domain concept.WebHome]] Scheme implies in terms of a role. 842 842 843 -This is all that is required for interoperability within acommunity.The importantpoint is thata community mustrecognise a specific Agencyashavingtheauthorityto define(%style="color:#e74c3c"%)concept(%%)roles and to maintainthese“role” (%style="color:#e74c3c" %)concepts(%%) in a [[concept scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]]togetherwith documentationonthemeaningoftheroleand anyrelevantprocessing implications.This will thenensurethere is interoperability betweensystemsthat understandtheuseofthese(% style="color:#e74c3c" %)concepts(%%).933 +This explicitly states that this Dimension is playing a role identified by the FREQ concept in the Cross-Domain Concept Scheme. Again the application needs to understand what FREQ in the Cross-Domain Concept Scheme implies in terms of a role. 844 844 845 - Notethat eachof the[[Components>>doc:sdmx:Glossary.Component.WebHome]](Data[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]],Primary[[Measure>>doc:sdmx:Glossary.Measure.WebHome]],[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],Time[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]])hasamandatoryidentityassociation((%style="color:#e74c3c"%)Concept(%%) Identity)andifthis(% style="color:#e74c3c" %)Concept(%%)alsoidentifies therole thenit ispossibletostate thisby935 +This is all that is required for interoperability within a community. The important point is that a community must recognise a specific Agency as having the authority to define concept roles and to maintain these “role” concepts in a concept scheme together with documentation on the meaning of the role and any relevant processing implications. This will then ensure there is interoperability between systems that understand the use of these concepts. 846 846 937 +Note that each of the Components (Data Attribute, Primary Measure, Dimension, Measure Dimension, Time Dimension) has a mandatory identity association (Concept Identity) and if this Concept also identifies the role then it is possible to state this by 938 + 847 847 == 7.5 SDMX Cross Domain Concept Scheme == 848 848 849 -All (% style="color:#e74c3c" %)concepts(%%)in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]Cross Domain[[Concept Scheme>>doc:sdmx:Glossary.Conceptscheme.WebHome]]are capable of playing a role and this scheme will contain all of the roles that were allowed at(% style="color:#e74c3c" %)version(%%)2.0 and will be maintained with new roles that are agreed at the(% style="color:#e74c3c" %)level(%%)of the community using the Cross Domain[[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]].941 +All concepts in the SDMX Cross Domain Concept Scheme are capable of playing a role and this scheme will contain all of the roles that were allowed at version 2.0 and will be maintained with new roles that are agreed at the level of the community using the Cross Domain Concept Scheme. 850 850 851 -The table below lists the (% style="color:#e74c3c" %)Concepts(%%)that need to be in this scheme either for compatibility with(% style="color:#e74c3c" %)version(%%)2.0 or because of requests for additional roles at(% style="color:#e74c3c" %)version(%%)2.1 which have been accepted.943 +The table below lists the Concepts that need to be in this scheme either for compatibility with version 2.0 or because of requests for additional roles at version 2.1 which have been accepted. 852 852 853 -Note that each of the [[Components>>doc:sdmx:Glossary.Component.WebHome]](Data[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]], Primary[[Measure>>doc:sdmx:Glossary.Measure.WebHome]],[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],[[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], Time[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]) has a mandatory identity association ((% style="color:#e74c3c" %)Concept(%%)Identity) and if this(% style="color:#e74c3c" %)Concept(%%)also identifies the role then it is possible to state this by means of the isRole[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]](isRole=true) Additional roles can still be specified by means of the +role association to additional(% style="color:#e74c3c" %)Concepts(%%)that identify the role.945 +Note that each of the Components (Data Attribute, Primary Measure, Dimension, Measure Dimension, Time Dimension) has a mandatory identity association (Concept Identity) and if this Concept also identifies the role then it is possible to state this by means of the isRole attribute (isRole=true) Additional roles can still be specified by means of the +role association to additional Concepts that identify the role. 854 854 855 855 = 8 Constraints = 856 856 857 857 == 8.1 Introduction == 858 858 859 -In this (% style="color:#e74c3c" %)version(%%)of[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]the (% style="color:#e74c3c" %)Constraints(%%)is a[[Maintainable Artefact>>doc:sdmx:Glossary.Maintainableartefact.WebHome]] can be associated to one or more of:951 +In this version of SDMX the Constraints is a Maintainable Artefact can be associated to one or more of: 860 860 861 861 * Data Structure Definition 862 -* [[Metadata Structure Definition>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]863 -* [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]864 -* [[Metadataflow>>doc:sdmx:Glossary.Metadataflow.WebHome]]865 -* [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]]866 -* [[Data Provider>>doc:sdmx:Glossary.Dataprovider.WebHome]](this is restricted to a[[Release Calendar>>doc:sdmx:Glossary.Releasepolicy - release calendar.WebHome]] (% style="color:#e74c3c" %)Constraint(%%))954 +* Metadata Structure Definition 955 +* Dataflow 956 +* Metadataflow 957 +* Provision Agreement 958 +* Data Provider (this is restricted to a Release Calendar Constraint) 867 867 * Simple or Queryable Datasources 868 868 869 -Note that regardless of the artifact to which the (% style="color:#e74c3c" %)Constraint(%%)is associated, it is constraining the contents of[[code lists>>doc:sdmx:Glossary.Codelist.WebHome]] in the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]towhich the constrained object is related. This does not apply, of course, to a[[Data Provider>>doc:sdmx:Glossary.Dataprovider.WebHome]]as the[[Data Provider>>doc:sdmx:Glossary.Dataprovider.WebHome]]can be associated, via the[[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]], to many DSDs. Hence the reason for the restriction on the type of(% style="color:#e74c3c" %)Constraint(%%)that can be attached to a[[Data Provider>>doc:sdmx:Glossary.Data provider.WebHome]].961 +Note that regardless of the artifact to which the Constraint is associated, it is constraining the contents of code lists in the DSD to which the constrained object is related. This does not apply, of course, to a Data Provider as the Data Provider can be associated, via the Provision Agreement, to many DSDs. Hence the reason for the restriction on the type of Constraint that can be attached to a Data Provider. 870 870 871 871 == 8.2 Types of Constraint == 872 872 873 -The (% style="color:#e74c3c" %)Constraint(%%)can be of one of two types:965 +The Constraint can be of one of two types: 874 874 875 -* Content (% style="color:#e74c3c" %)constraint876 -* Attachable (% style="color:#e74c3c" %)constraint967 +* Content constraint 968 +* Attachable constraint 877 877 878 -The attachable (% style="color:#e74c3c" %)constraint(%%)is used to define “cube slices” which identify sub sets of data in terms of[[series keys>>doc:sdmx:Glossary.Serieskey.WebHome]] or[[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]values. The purpose of this is to enable metadata to be attached to the(% style="color:#e74c3c" %)constraint(%%), and thereby to the cube slices defined in the(% style="color:#e74c3c" %)Constraint(%%). The metadata can be attached via the “[[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]]” mechanism –[[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]]and[[Metadata Set>>doc:sdmx:Glossary.Metadataset.WebHome]]– or via a Group in the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]].Below is snippet of the schema for a[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] that shows the constructs that enable the(% style="color:#e74c3c" %)Constraint(%%)to referenced from a Group in a[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].970 +The attachable constraint is used to define “cube slices” which identify sub sets of data in terms of series keys or dimension values. The purpose of this is to enable metadata to be attached to the constraint, and thereby to the cube slices defined in the Constraint. The metadata can be attached via the “reference metadata” mechanism – MSD and Metadata Set – or via a Group in the DSD. Below is snippet of the schema for a DSD that shows the constructs that enable the Constraint to referenced from a Group in a DSD. 879 879 880 880 [[image:1747836776695-806.jpeg]] 881 881 882 882 **Figure 9: Extract from the SDMX-ML Schema showing reference to Attachment Constraint** 883 883 884 -For the Content (% style="color:#e74c3c" %)Constraint(%%)specific “inheritance” rules apply and these are detailed below.976 +For the Content Constraint specific “inheritance” rules apply and these are detailed below. 885 885 886 886 == 8.3 Rules for a Content Constraint == 887 887 888 -=== 8.3.1 Scope of a Content Constraint === 980 +=== 8.3.1 Scope of a Content Constraint === 889 889 890 -A Content (% style="color:#e74c3c" %)Constraint(%%)is used specify the content of a data or metadata source in terms of the[[component>>doc:sdmx:Glossary.Component.WebHome]]values or the keys.982 +A Content Constraint is used specify the content of a data or metadata source in terms of the component values or the keys. 891 891 892 -In terms of data the [[components>>doc:sdmx:Glossary.Component.WebHome]]are:984 +In terms of data the components are: 893 893 894 -* [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]895 -* [[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]896 -* Time [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]897 -* Data [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]898 -* Primary [[Measure>>doc:sdmx:Glossary.Measure.WebHome]]986 +* Dimension 987 +* Measure Dimension 988 +* Time Dimension 989 +* Data Attribute 990 +* Primary Measure 899 899 900 -And the keys are the content of the KeyDescriptor – i.e. the [[series keys>>doc:sdmx:Glossary.Serieskey.WebHome]]composed, for each key, by a value for each[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]and[[Measure>>doc:sdmx:Glossary.Measure.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]992 +And the keys are the content of the KeyDescriptor – i.e. the series keys composed, for each key, by a value for each Dimension and Measure Dimension 901 901 902 -In terms of [[reference metadata>>doc:sdmx:Glossary.Referencemetadata.WebHome]] the[[components>>doc:sdmx:Glossary.Component.WebHome]]are:994 +In terms of reference metadata the components are: 903 903 904 904 * Target Object which is one of: 905 -** Key Descriptor Values o [[Data Set>>doc:sdmx:Glossary.Dataset.WebHome]]oReport Period997 +** Key Descriptor Values o Data Set o Report Period 906 906 ** IdentifiableObject 907 -* Metadata [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]999 +* Metadata Attribute 908 908 909 -The “key” is therefore the combination of the Target Objects that are defined for the Metadata Target. 1001 +The “key” is therefore the combination of the Target Objects that are defined for the Metadata Target. 910 910 911 -For a (% style="color:#e74c3c" %)Constraint(%%)based on a[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] the Content(% style="color:#e74c3c" %)Constraint(%%)can reference one or more of:1003 +For a Constraint based on a DSD the Content Constraint can reference one or more of: 912 912 913 913 * Data Structure Definition 914 -* [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]915 -* [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]]1006 +* Dataflow 1007 +* Provision Agreement 916 916 917 -For a (% style="color:#e74c3c" %)Constraint(%%)based on an[[MSD>>doc:sdmx:Glossary.Metadatastructure definition.WebHome]] the Content(% style="color:#e74c3c" %)Constraint(%%)can reference one or more of:1009 +For a Constraint based on an MSD the Content Constraint can reference one or more of: 918 918 919 -* [[Metadata Structure Definition>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]920 -* [[Metadataflow>>doc:sdmx:Glossary.Metadataflow.WebHome]]921 -* [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]]1011 +* Metadata Structure Definition 1012 +* Metadataflow 1013 +* Provision Agreement 922 922 923 -Furthermore, there can be more than one Content (% style="color:#e74c3c" %)Constraint(%%)specified for a specific object e.g. more than one(% style="color:#e74c3c" %)Constraint(%%)for a specific[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].1015 +Furthermore, there can be more than one Content Constraint specified for a specific object e.g. more than one Constraint for a specific DSD. 924 924 925 -In view of the flexibility of (% style="color:#e74c3c" %)constraints(%%)attachment, clear rules on their usage are required. These are elaborated below.1017 +In view of the flexibility of constraints attachment, clear rules on their usage are required. These are elaborated below. 926 926 927 -=== 8.3.2 Multiple Content Constraints === 1019 +=== 8.3.2 Multiple Content Constraints === 928 928 929 -There can be many Content (% style="color:#e74c3c" %)Constraints(%%)for any Constrainable[[Artefact>>doc:sdmx:Glossary.Artefact.WebHome]](e.g.[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]), subject to the following restrictions:1021 +There can be many Content Constraints for any Constrainable Artefact (e.g. DSD), subject to the following restrictions: 930 930 931 - ====8.3.2.1 Cube Region====1023 +**8.3.2.1 Cube Region** 932 932 933 -1. The (% style="color:#e74c3c" %)constraint(%%)can contain multiple[[Member Selections>>doc:sdmx:Glossary.Memberselection.WebHome]](e.g.[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]) but:934 -1. A specific [[Member Selection>>doc:sdmx:Glossary.Memberselection.WebHome]](e.g.[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]FREQ) can only be contained in one Content(% style="color:#e74c3c" %)Constraint(%%)for any one attached object (e.g. a specific[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] or specific[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]])1025 +1. The constraint can contain multiple Member Selections (e.g. Dimension) but: 1026 +1. A specific Member Selection (e.g. Dimension FREQ) can only be contained in one Content Constraint for any one attached object (e.g. a specific DSD or specific Dataflow) 935 935 936 - ====8.3.2.2 Key Set====1028 +**8.3.2.2 Key Set** 937 937 938 -Key Sets will be processed in the order they appear in the (% style="color:#e74c3c" %)Constraint(%%)and wildcards can be used (e.g. any key position not reference explicitly is deemed to be “all values”). As the Key Sets can be “included” or “excluded” it is recommended that Key Sets with wildcards are declared before KeySets with specific[[series keys>>doc:sdmx:Glossary.Serieskey.WebHome]].This will minimize the risk that keys are inadvertently included or excluded.1030 +Key Sets will be processed in the order they appear in the Constraint and wildcards can be used (e.g. any key position not reference explicitly is deemed to be “all values”). As the Key Sets can be “included” or “excluded” it is recommended that Key Sets with wildcards are declared before KeySets with specific series keys. This will minimize the risk that keys are inadvertently included or excluded. 939 939 940 -=== 8.3.3 Inheritance of a Content Constraint === 1032 +=== 8.3.3 Inheritance of a Content Constraint === 941 941 942 - ====8.3.3.1 Attachment levels of a Content Constraint====1034 +**8.3.3.1 Attachment levels of a Content Constraint** 943 943 944 -There are three (% style="color:#e74c3c" %)levels(%%)of(% style="color:#e74c3c" %)constraint(%%)attachment for which these inheritance rules apply:1036 +There are three levels of constraint attachment for which these inheritance rules apply: 945 945 946 -* [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]/[[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]] – top (% style="color:#e74c3c" %)level 947 -** [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]/[[Metadataflow>>doc:sdmx:Glossary.Metadataflow.WebHome]] – second (% style="color:#e74c3c" %)level 948 -*** [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]] – third (% style="color:#e74c3c" %)level 1038 + DSD/MSD – top level o Dataflow/Metadataflow – second level 949 949 950 - Notethat these rules do not apply to the Simple Datasoucre or Queryable Datasource: the Content (% style="color:#e74c3c" %)Constraint(%%)(s) attached to these [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] are resolved for this [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] only and do not take into account (% style="color:#e74c3c" %)Constraints(%%) attached to other [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] (e.g. [[Provision Agreement>>doc:sdmx:Glossary.Provisionagreement.WebHome]].[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], [[DSD>>doc:sdmx:Glossary.Datastructuredefinition.WebHome]]).1040 +§ Provision Agreement – third level 951 951 952 - It is notnecessaryforaContent (%style="color:#e74c3c"%)Constraint(%%)tobe attachedtohigher(% style="color:#e74c3c"%)level(%%)artifact.e.g.itis valid tohaveaContent(% style="color:#e74c3c" %)Constraint(%%)fora[[ProvisionAgreement>>doc:sdmx:Glossary.Provisionagreement.WebHome]]wherethereare no(%style="color:#e74c3c"%)constraints(%%)attached therelevant[[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]or [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].1042 +Note that these rules do not apply to the Simple Datasoucre or Queryable Datasource: the Content Constraint(s) attached to these artefacts are resolved for this artefact only and do not take into account Constraints attached to other artefacts (e.g. Provision Agreement. Dataflow, DSD). 953 953 954 - ====8.3.3.2Cascade rules forprocessingConstraints====1044 +It is not necessary for a Content Constraint to be attached to higher level artifact. e.g. it is valid to have a Content Constraint for a Provision Agreement where there are no constraints attached the relevant dataflow or DSD. 955 955 956 - The processing of the (% style="color:#e74c3c" %)constraints(%%) on either [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]/[[Metadataflow>>doc:sdmx:Glossary.Metadataflow.WebHome]]or [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]] must take into account the (% style="color:#e74c3c" %)constraints(%%)declaredat higher(% style="color:#e74c3c" %)levels(%%). The rules forthe lower(% style="color:#e74c3c" %)level(%%) (%style="color:#e74c3c" %)constraints(%%)(attached to[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]/ [[Metadataflow>>doc:sdmx:Glossary.Metadataflow.WebHome]] and [[ProvisionAgreement>>doc:sdmx:Glossary.Provision agreement.WebHome]]) are detailed below.1046 +**8.3.3.2 Cascade rules for processing Constraints** 957 957 958 - Note that therecan beasituationwherea (% style="color:#e74c3c"%)constraint(%%) isspecifiedatalower (% style="color:#e74c3c" %)level(%%) beforea(% style="color:#e74c3c" %)constraint(%%) is specifiedatahigher (% style="color:#e74c3c"%)level(%%). Therefore,it ispossiblethat a higher(% style="color:#e74c3c" %)level(%%)(%style="color:#e74c3c"%)constraint(%%) makesa lower(% style="color:#e74c3c"%)level(%%) (% style="color:#e74c3c"%)constraint(%%)invalid. [[SDMX>>doc:sdmx:Glossary.Statisticaldataand metadataexchange.WebHome]]makesnorules onhow such a conflictshould behandledwhen processingthe (% style="color:#e74c3c"%)constraint(%%)forattachment.However,the cascade rulesonevaluating (% style="color:#e74c3c"%)constraints(%%)forusage are clear - the higher (% style="color:#e74c3c" %)level(%%) (%style="color:#e74c3c" %)constraint(%%)takes precedence in any conflictsthatresultin a less restrictive specification at thelower(% style="color:#e74c3c" %)level(%%).1048 +The processing of the constraints on either Dataflow/Metadataflow or Provision Agreement must take into account the constraints declared at higher levels. The rules for the lower level constraints (attached to Dataflow/ Metadataflow and Provision Agreement) are detailed below. 959 959 960 - ====8.3.3.3CubeRegion====1050 +Note that there can be a situation where a constraint is specified at a lower level before a constraint is specified at a higher level. Therefore, it is possible that a higher level constraint makes a lower level constraint invalid. SDMX makes no rules on how such a conflict should be handled when processing the constraint for attachment. However, the cascade rules on evaluating constraints for usage are clear - the higher level constraint takes precedence in any conflicts that result in a less restrictive specification at the lower level. 961 961 962 -1. It is not necessary to have a (% style="color:#e74c3c" %)constraint(%%) on the higher (% style="color:#e74c3c" %)level(%%) artifact (e.g. [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] referenced by the [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]) but if there is such a (% style="color:#e74c3c" %)constraint(%%) at the higher (% style="color:#e74c3c" %)level(%%)(s) then: 963 -a. The lower (% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%) cannot be less restrictive than the (% style="color:#e74c3c" %)constraint(%%) specified for the same [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] (e.g. [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]) at the next higher (% style="color:#e74c3c" %)level(%%) which (% style="color:#e74c3c" %)constraints(%%) that [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] (e.g. if the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] FREQ is constrained to A, Q in a [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] then the (% style="color:#e74c3c" %)constraint(%%) at the [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] or [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]] cannot be A, Q, M or even just M – it can only further constrain A,Q). 964 -b. The (% style="color:#e74c3c" %)constraint(%%) at the lower (% style="color:#e74c3c" %)level(%%) for any one [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] further constrains the content for the same [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] at the higher (% style="color:#e74c3c" %)level(%%)(s). 965 -1. Any [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] which is not referenced in a Content (% style="color:#e74c3c" %)Constraint(%%) is deemed to be constrained according to the Content (% style="color:#e74c3c" %)Constraint(%%) specified at the next higher (% style="color:#e74c3c" %)level(%%) which (% style="color:#e74c3c" %)constraints(%%) that [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]]. 966 -1. If there is a conflict when resolving the (% style="color:#e74c3c" %)constraint(%%) in terms of a lower-(% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%) being less restrictive than a higher-(% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%) then the (% style="color:#e74c3c" %)constraint(%%) at the higher-(% style="color:#e74c3c" %)level(%%) is used. 1052 +**8.3.3.3 Cube Region** 967 967 968 -Note that it is possible for a Content (% style="color:#e74c3c" %)Constraint(%%) at a higher (% style="color:#e74c3c" %)level(%%) to constrain, say, four [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] in a single (% style="color:#e74c3c" %)constraint(%%), and a Content (% style="color:#e74c3c" %)Constraint(%%) at a lower (% style="color:#e74c3c" %)level(%%) to constrain the same four in two, three, or four Content (% style="color:#e74c3c" %)Constraints(%%). 1054 +1. It is not necessary to have a constraint on the higher level artifact (e.g. DSD referenced by the Dataflow) but if there is such a constraint at the higher level(s) then: 1055 +11. The lower level constraint cannot be less restrictive than the constraint specified for the same Member Selection (e.g. Dimension) at the next higher level which constraints that Member Selection (e.g. if the Dimension FREQ is constrained to A, Q in a DSD then the constraint at the Dataflow or Provision Agreement cannot be A, Q, M or even just M – it can only further constrain A,Q). 1056 +11. The constraint at the lower level for any one Member Selection further constrains the content for the same Member Selection at the higher level(s). 1057 +1. Any Member Selection which is not referenced in a Content Constraint is deemed to be constrained according to the Content Constraint specified at the next higher level which constraints that Member Selection. 1058 +1. If there is a conflict when resolving the constraint in terms of a lower-level constraint being less restrictive than a higher-level constraint then the constraint at the higher-level is used. 969 969 970 - ====8.3.3.4KeySet====1060 +Note that it is possible for a Content Constraint at a higher level to constrain, say, four Dimensions in a single constraint, and a Content Constraint at a lower level to constrain the same four in two, three, or four Content Constraints. 971 971 972 -1. It is not necessary to have a (% style="color:#e74c3c" %)constraint(%%) on the higher (% style="color:#e74c3c" %)level(%%) [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] (e.g. [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] referenced by the [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]) but if there is such a (% style="color:#e74c3c" %)constraint(%%) at the higher (% style="color:#e74c3c" %)level(%%)(s) then: 973 -a. The lower (% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%) cannot be less restrictive than the (% style="color:#e74c3c" %)constraint(%%) specified at the higher (% style="color:#e74c3c" %)level(%%). 974 -b. The (% style="color:#e74c3c" %)constraint(%%) at the lower (% style="color:#e74c3c" %)level(%%) for any one [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] further constrains the keys specified at the higher (% style="color:#e74c3c" %)level(%%)(s). 975 -1. Any [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]] which is not referenced in a Content (% style="color:#e74c3c" %)Constraint(%%) is deemed to be constrained according to the Content (% style="color:#e74c3c" %)Constraint(%%) specified at the next higher (% style="color:#e74c3c" %)level(%%) which (% style="color:#e74c3c" %)constraints(%%) that [[Member Selection>>doc:sdmx:Glossary.Member selection.WebHome]]. 976 -1. If there is a conflict when resolving the keys in the (% style="color:#e74c3c" %)constraint(%%) at two (% style="color:#e74c3c" %)levels(%%), in terms of a lower-(% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%) being less restrictive than a higher-(% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%), then the offending keys specified at the lower (% style="color:#e74c3c" %)level(%%) are not deemed part of the (% style="color:#e74c3c" %)constraint(%%). 1062 +**8.3.3.4 Key Set** 977 977 978 -Note that a Key in a Key Set can have wildcarded [[Components>>doc:sdmx:Glossary.Component.WebHome]]. For instance the (% style="color:#e74c3c" %)constraint(%%) may simply constrain the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] FREQ to “A”, and all keys where the FREQ=A are therefore valid. 1064 +1. It is not necessary to have a constraint on the higher level artefact (e.g. DSD referenced by the Dataflow) but if there is such a constraint at the higher level(s) then: 1065 +11. The lower level constraint cannot be less restrictive than the constraint specified at the higher level. 1066 +11. The constraint at the lower level for any one Member Selection further constrains the keys specified at the higher level(s). 1067 +1. Any Member Selection which is not referenced in a Content Constraint is deemed to be constrained according to the Content Constraint specified at the next higher level which constraints that Member Selection. 1068 +1. If there is a conflict when resolving the keys in the constraint at two levels, in terms of a lower-level constraint being less restrictive than a higher-level constraint, then the offending keys specified at the lower level are not deemed part of the constraint. 979 979 980 - Thefollowinglogicexplainshow the inheritancemechanismworks.Note thatthisisconceptuallogicand actualsystemsmay differinthewaythis is implemented.1070 +Note that a Key in a Key Set can have wildcarded Components. For instance the constraint may simply constrain the Dimension FREQ to “A”, and all keys where the FREQ=A are therefore valid. 981 981 982 -1. Determine all possible keys that are valid at the higher (% style="color:#e74c3c" %)level(%%). 983 -1. These keys are deemed to be inherited by the lower (% style="color:#e74c3c" %)level(%%) constrained object, subject to the (% style="color:#e74c3c" %)constraints(%%) specified at the lower (% style="color:#e74c3c" %)level(%%). 984 -1. Determine all possible keys that are possible using the (% style="color:#e74c3c" %)constraints(%%) specified at the lower (% style="color:#e74c3c" %)level(%%). 985 -1. At the lower (% style="color:#e74c3c" %)level(%%) inherit all keys that match with the higher (% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%). 986 -1. If there are keys in the lower (% style="color:#e74c3c" %)level(%%) (% style="color:#e74c3c" %)constraint(%%) that are not inherited then the key is invalid (i.e. it is less restrictive). 1072 +The following logic explains how the inheritance mechanism works. Note that this is conceptual logic and actual systems may differ in the way this is implemented. 987 987 988 -=== 8.3.4 Constraints Examples === 1074 +1. Determine all possible keys that are valid at the higher level. 1075 +1. These keys are deemed to be inherited by the lower level constrained object, subject to the constraints specified at the lower level. 1076 +1. Determine all possible keys that are possible using the constraints specified at the lower level. 1077 +1. At the lower level inherit all keys that match with the higher level constraint. 1078 +1. If there are keys in the lower level constraint that are not inherited then the key is invalid (i.e. it is less restrictive). 989 989 1080 +**8.3.4 Constraints Examples** 1081 + 990 990 The following scenario is used. 991 991 992 - __[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]__1084 +=== DSD === 993 993 994 -This contains the following [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]:1086 +This contains the following Dimensions: 995 995 996 996 * GEO – Geography 997 997 * SEX – Sex 998 -* [[AGE>>doc:sdmx:Glossary.Age.WebHome]]–[[Age>>doc:sdmx:Glossary.Age.WebHome]]1090 +* AGE – Age 999 999 * CAS – Current Activity Status 1000 1000 1001 -In the [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] common[[code lists>>doc:sdmx:Glossary.Codelist.WebHome]]are used and the requirement is to restrict these at various(% style="color:#e74c3c" %)levels(%%)to specify the actual[[code>>doc:sdmx:Glossary.Code.WebHome]]that are valid for the object to which the Content(% style="color:#e74c3c" %)Constraint(%%)is attached.1093 +In the DSD common code lists are used and the requirement is to restrict these at various levels to specify the actual code that are valid for the object to which the Content Constraint is attached. 1002 1002 1003 -[[image:1747855493531-357.png]] 1004 1004 1005 -**Figure 10: Example Scenario for (% style="color:#e74c3c" %)Constraints(%%)** 1096 +|((( 1097 + 1098 +))) 1006 1006 1007 -(% style="color:#e74c3c" %)Constraints(%%) are declared as follows: 1100 +|((( 1101 + 1102 +))) 1008 1008 1009 -[[image:1747855462293-368.png]] 1104 +|((( 1105 + 1106 +))) 1010 1010 1011 -**Figure 11: Example Content (% style="color:#e74c3c" %)Constraints(%%)** 1108 +|((( 1109 +**Figure** 1110 +))) 1012 1012 1112 +|((( 1113 +**10** 1114 +))) 1115 + 1116 +|((( 1117 +**:** 1118 +))) 1119 + 1120 +|((( 1121 +**~ Example Sce** 1122 +))) 1123 + 1124 +|((( 1125 +**nario for Constraints** 1126 +))) 1127 + 1128 +|((( 1129 +**~ ** 1130 +))) 1131 + 1132 + 1133 + 1134 +Constraints are declared as follows: 1135 + 1136 + 1137 +|((( 1138 + 1139 +))) 1140 + 1141 +|((( 1142 + 1143 +))) 1144 + 1145 +|((( 1146 + 1147 +))) 1148 + 1149 +|((( 1150 +**Figure** 1151 +))) 1152 + 1153 +|((( 1154 +**11** 1155 +))) 1156 + 1157 +|((( 1158 +**:** 1159 +))) 1160 + 1161 +|((( 1162 +**~ Example Content Constraints** 1163 +))) 1164 + 1165 +|((( 1166 +**~ ** 1167 +))) 1168 + 1169 + 1170 + 1013 1013 **Notes:** 1014 1014 1015 -1. [[AGE>>doc:sdmx:Glossary.Age.WebHome]] is constrained for the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] and is further restricted for the [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] CENSUS_CUBE1. 1016 -1. The same (% style="color:#e74c3c" %)Constraint(%%) applies to both Provision Agreements. 1173 +1. AGE is constrained for the DSD and is further restricted for the Dataflow 1017 1017 1175 +CENSUS_CUBE1. 1176 + 1177 +1. The same Constraint applies to both Provision Agreements. 1178 + 1018 1018 The cascade rules elaborated above result as follows: 1019 1019 1020 - __[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]]__1181 +DSD 1021 1021 1022 -~1. Constrained by eliminating [[code>>doc:sdmx:Glossary.Code.WebHome]]001 from the[[code list>>doc:sdmx:Glossary.Codelist.WebHome]]for the[[AGE>>doc:sdmx:Glossary.Age.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]].1183 +~1. Constrained by eliminating code 001 from the code list for the AGE Dimension. 1023 1023 1024 - __[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]CENSUS_CUBE1__1185 +=== Dataflow CENSUS_CUBE1 === 1025 1025 1026 -1. Constrained by restricting the [[code list>>doc:sdmx:Glossary.Codelist.WebHome]]for the[[AGE>>doc:sdmx:Glossary.Age.WebHome]][[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]to[[codes>>doc:sdmx:Glossary.Code.WebHome]]002 and 003(note that this is a more restrictive(% style="color:#e74c3c" %)constraint(%%)than that declared for the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]which specifies all[[codes>>doc:sdmx:Glossary.Code.WebHome]]except[[code>>doc:sdmx:Glossary.Code.WebHome]]001).1027 -1. Restricts the CAS [[codes>>doc:sdmx:Glossary.Code.WebHome]]to 003 and 004.1187 +1. Constrained by restricting the code list for the AGE Dimension to codes 002 and 003(note that this is a more restrictive constraint than that declared for the DSD which specifies all codes except code 001). 1188 +1. Restricts the CAS codes to 003 and 004. 1028 1028 1029 - __[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]CENSUS_CUBE2__1190 +=== Dataflow CENSUS_CUBE2 === 1030 1030 1031 -1. Restricts the [[code list>>doc:sdmx:Glossary.Codelist.WebHome]]for the CAS[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]to[[codes>>doc:sdmx:Glossary.Code.WebHome]]TOT and NAP.1032 -1. Inherits the [[AGE>>doc:sdmx:Glossary.Age.WebHome]](% style="color:#e74c3c" %)constraint(%%)applied at the(% style="color:#e74c3c" %)level(%%)of the[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].1192 +1. Restricts the code list for the CAS Dimension to codes TOT and NAP. 1193 +1. Inherits the AGE constraint applied at the level of the DSD. 1033 1033 1034 - __Provision Agreements CENSUS_CUBE1_IT__1195 +=== Provision Agreements CENSUS_CUBE1_IT === 1035 1035 1036 -1. Restricts the [[codes>>doc:sdmx:Glossary.Code.WebHome]]for the GEO[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]to IT and its children.1037 -1. Inherits the (% style="color:#e74c3c" %)constraints(%%)from[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]CENSUS_CUBE1 for the[[AGE>>doc:sdmx:Glossary.Age.WebHome]]and CAS[[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]].1197 +1. Restricts the codes for the GEO Dimension to IT and its children. 1198 +1. Inherits the constraints from Dataflow CENSUS_CUBE1 for the AGE and CAS Dimensions. 1038 1038 1039 - __Provision Agreements CENSUS_CUBE2_IT__1200 +=== Provision Agreements CENSUS_CUBE2_IT === 1040 1040 1041 -1. Restricts the [[codes>>doc:sdmx:Glossary.Code.WebHome]]for the GEO[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]to IT and its children.1042 -1. Inherits the (% style="color:#e74c3c" %)constraints(%%)from[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]CENSUS_CUBE2 for the CAS[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]].1043 -1. Inherits the [[AGE>>doc:sdmx:Glossary.Age.WebHome]](% style="color:#e74c3c" %)constraint(%%)applied at the(% style="color:#e74c3c" %)level(%%)of the[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].1202 +1. Restricts the codes for the GEO Dimension to IT and its children. 1203 +1. Inherits the constraints from Dataflow CENSUS_CUBE2 for the CAS Dimension. 1204 +1. Inherits the AGE constraint applied at the level of the DSD. 1044 1044 1045 -The (% style="color:#e74c3c" %)constraints(%%)are defined as follows:1206 +The constraints are defined as follows: 1046 1046 1047 - __[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] (% style="color:#e74c3c" %)Constraint(%%)__1208 +=== DSD Constraint === 1048 1048 1049 1049 [[image:1747836776698-720.jpeg]] 1050 1050 1051 - __[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]](% style="color:#e74c3c" %)Constraints(%%)__1212 +=== Dataflow Constraints === 1052 1052 1053 1053 [[image:1747836776701-360.jpeg]] 1054 1054 1055 -[[image:1747836776707-834.jpeg]] 1216 +=== [[image:1747836776707-834.jpeg]] === 1056 1056 1057 - __[[Provision Agreement>>doc:sdmx:Glossary.Provisionagreement.WebHome]] (% style="color:#e74c3c" %)Constraint(%%)__1218 +=== Provision Agreement Constraint === 1058 1058 1059 1059 [[image:1747836776710-262.jpeg]] 1060 1060 ... ... @@ -1062,115 +1062,122 @@ 1062 1062 1063 1063 == 9.1 Scope == 1064 1064 1065 -The scope of this section is to define both best practices and mandatory behaviour for specific aspects of transformation between different formats of [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].1226 +The scope of this section is to define both best practices and mandatory behaviour for specific aspects of transformation between different formats of SDMX. 1066 1066 1067 1067 == 9.2 Groups and Dimension Groups == 1068 1068 1069 -=== 9.2.1 Issue === 1230 +=== 9.2.1 Issue === 1070 1070 1071 - (% style="color:#e74c3c" %)Version(%%)2.1 introduces a more granular mechanism for specifying the relationship between a Data[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]and the[[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]to which the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]applies. The technical construct for this is the[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]Group. This[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]Group has no direct equivalent in(% style="color:#e74c3c" %)versions(%%)2.0 and 1.0 and so the application transforming data from a(% style="color:#e74c3c" %)version(%%)2.1[[data set>>doc:sdmx:Glossary.Dataset.WebHome]]toa(% style="color:#e74c3c" %)version(%%)2.0 or(% style="color:#e74c3c" %)version(%%)1.0[[data set>>doc:sdmx:Glossary.Dataset.WebHome]] must decide to which construct the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value, whose[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]is declared in a[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]Group, should be attached. The closest construct is the “Series”[[attachment level>>doc:sdmx:Glossary.Attachmentlevel.WebHome]]and in many cases this is the correct construct to use.1232 +Version 2.1 introduces a more granular mechanism for specifying the relationship between a Data Attribute and the Dimensions to which the attribute applies. The technical construct for this is the Dimension Group. This Dimension Group has no direct equivalent in versions 2.0 and 1.0 and so the application transforming data from a version 2.1 data set to a version 2.0 or version 1.0 data set must decide to which construct the attribute value, whose Attribute is declared in a Dimension Group, should be attached. The closest construct is the “Series” attachment level and in many cases this is the correct construct to use. 1072 1072 1073 -However, there is one case where the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]MUST be attached to a Group in the(% style="color:#e74c3c" %)version(%%)2.0 and 1.0 message. The conditions of this case are:1234 +However, there is one case where the attribute MUST be attached to a Group in the version 2.0 and 1.0 message. The conditions of this case are: 1074 1074 1075 -1. A Group is defined in the [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]with exactly the same[[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]as a[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]Group in the same[[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].1076 -1. The [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]is defined in the[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]with an[[Attribute Relationship>>doc:sdmx:Glossary.Attributerelationship.WebHome]]to the[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]Group. This[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]is NOT defined as having an[[Attribute Relationship>>doc:sdmx:Glossary.Attributerelationship.WebHome]]to the Group.1236 +1. A Group is defined in the DSD with exactly the same Dimensions as a Dimension Group in the same DSD. 1237 +1. The Attribute is defined in the DSD with an Attribute Relationship to the Dimension Group. This attribute is NOT defined as having an Attribute Relationship to the Group. 1077 1077 1078 1078 === 9.2.2 Structural Metadata === 1079 1079 1080 -If the conditions defined in 9.2.1are true then on conversion to a (% style="color:#e74c3c" %)version(%%)2.0 or 1.0[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]](Key Family) the[[Component>>doc:sdmx:Glossary.Component.WebHome]]/[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]].attachmentLevel must be set to “Group” and the[[Component>>doc:sdmx:Glossary.Component.WebHome]]/[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]/AttachmentGroup” is used to identify the Group. Note that under rule(1) in 1.2.1 this group will have been defined in the V 2.1[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]]and so will be present in the V 2.0 transformation.1241 +If the conditions defined in 9.2.1are true then on conversion to a version 2.0 or 1.0 DSD (Key Family) the Component/Attribute.attachmentLevel must be set to “Group” and the Component/Attribute/AttachmentGroup” is used to identify the Group. Note that under rule(1) in 1.2.1 this group will have been defined in the V 2.1 DSD and so will be present in the V 2.0 transformation. 1081 1081 1082 -=== 9.2.3 Data === 1243 +=== 9.2.3 Data === 1083 1083 1084 -If the conditions defined in 9.2.1are true then, on conversion from a 2.1 [[data set>>doc:sdmx:Glossary.Dataset.WebHome]]toa 2.0 or 1.0[[dataset>>doc:sdmx:Glossary.Dataset.WebHome]] the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value will be placed in the relevant . If these conditions are not true then the[[attribute>>doc:sdmx:Glossary.Attribute.WebHome]]value will be placed in the .1245 +If the conditions defined in 9.2.1are true then, on conversion from a 2.1 data set to a 2.0 or 1.0 dataset the attribute value will be placed in the relevant <Group>. If these conditions are not true then the attribute value will be placed in the <Series>. 1085 1085 1086 1086 === 9.2.4 Compact Schema === 1087 1087 1088 -If the conditions defined in 9.2.1are true then the Compact Schema must be generated with the Group present and the [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]](s) present in that group definition.1249 +If the conditions defined in 9.2.1are true then the Compact Schema must be generated with the Group present and the Attribute(s) present in that group definition. 1089 1089 1090 1090 = 10 Validation and Transformation Language (VTL) = 1091 1091 1092 1092 == 10.1 Introduction == 1093 1093 1094 -The [[Validation and Transformation Language>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]([[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]]) supports the definition of Transformations, which are algorithms to calculate new data starting from already existing ones[[(% class="wikiinternallink wikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[4~]^^>>path:#_ftn4]](%%). The purpose of the[[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]] inthe[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] context is to enable the:1255 +The Validation and Transformation Language (VTL) supports the definition of Transformations, which are algorithms to calculate new data starting from already existing ones[[(% class="wikiinternallink wikiinternallink" %)^^~[4~]^^>>path:#_ftn4]](%%). The purpose of the VTL in the SDMX context is to enable the: 1095 1095 1096 -* definition of validation and transformation algorithms, in order to specify how to calculate new data from existing ones; 1097 -* exchange of the definition of [[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]] algorithms, also together the definition of the data structures of the involved data (for example, exchange the data structures of a reporting framework together with the validation rules to be applied, exchange the input and output data structures of a calculation task together with the[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]Transformations describing the calculation algorithms);1098 -* compilation and execution of [[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]] algorithms, either interpreting the[[VTL>>doc:sdmx:Glossary.Validationandtransformationlanguage.WebHome]] transformationsor translating them in whatever other computer language is deemed as appropriate.1257 +* definition of validation and transformation algorithms, in order to specify how to calculate new data from existing ones; 1258 +* exchange of the definition of VTL algorithms, also together the definition of the data structures of the involved data (for example, exchange the data structures of a reporting framework together with the validation rules to be applied, exchange the input and output data structures of a calculation task together with the VTL Transformations describing the calculation algorithms); 1259 +* compilation and execution of VTL algorithms, either interpreting the VTL transformations or translating them in whatever other computer language is deemed as appropriate. 1099 1099 1100 -It is important to note that the [[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]has its own information model (IM), derived from the Generic Statistical Information Model (GSIM) and described in the[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]User Guide. The[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]IM is designed to be compatible with more standards, like[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], DDI (Data Documentation Initiative) and GSIM, and includes the model[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]that can be manipulated (inputs and/or outputs of transformations, e.g. “[[Data Set>>doc:sdmx:Glossary.Data set.WebHome]]”, “Data Structure”) and the model[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]that allow the definition of the transformation algorithms (e.g. “Transformation”, “Transformation Scheme”).1261 +It is important to note that the VTL has its own information model (IM), derived from the Generic Statistical Information Model (GSIM) and described in the VTL User Guide. The VTL IM is designed to be compatible with more standards, like SDMX, DDI (Data Documentation Initiative) and GSIM, and includes the model artefacts that can be manipulated (inputs and/or outputs of transformations, e.g. “Data Set”, “Data Structure”) and the model artefacts that allow the definition of the transformation algorithms (e.g. “Transformation”, “Transformation Scheme”). 1101 1101 1102 -The [[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]]languagecan be applied to[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]by(% style="color:#e74c3c" %)mapping(%%)the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]IM model[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]to the model[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]that[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]can manipulate. Thus, the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]can be used in[[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]] asinputs and/or outputs of transformations. It is important to be aware that the[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]do not always have the same names in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata andmetadata exchange.WebHome]] and [[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]IMs, nor do they always have the same meaning. The more evident example is given by the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[Dataset>>doc:sdmx:Glossary.Dataset.WebHome]]and the[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]“[[Data Set>>doc:sdmx:Glossary.Data set.WebHome]]”, which do not correspond one another: as a matter of fact, the[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]“[[Data Set>>doc:sdmx:Glossary.Data set.WebHome]]”(% style="color:#e74c3c" %)maps(%%)to the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]“[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]”, while the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]“[[Dataset>>doc:sdmx:Glossary.Data set.WebHome]]” has no explicit(% style="color:#e74c3c" %)mapping(%%)to[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]](such an abstraction is not needed in the definition of[[VTL>>doc:sdmx:Glossary.Validationandtransformationlanguage.WebHome]] transformations). A[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]“[[Dataset>>doc:sdmx:Glossary.Data set.WebHome]]”, however, is an instance of a[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]“[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]” and can be the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]on which the[[VTL>>doc:sdmx:Glossary.Validationandtransformationlanguage.WebHome]] transformationsare executed (i.e., the transformations are defined on[[Dataflows>>doc:sdmx:Glossary.Dataflow.WebHome]]and are applied to[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]instances that can be Datasets).1263 +The VTL language can be applied to SDMX artefacts by mapping the SDMX IM model artefacts to the model artefacts that VTL can manipulate. Thus, the SDMX artefacts can be used in VTL as inputs and/or outputs of transformations. It is important to be aware that the artefacts do not always have the same names in the SDMX and VTL IMs, nor do they always have the same meaning. The more evident example is given by the SDMX Dataset and the VTL “Data Set”, which do not correspond one another: as a matter of fact, the VTL “Data Set” maps to the SDMX “Dataflow”, while the SDMX “Dataset” has no explicit mapping to VTL (such an abstraction is not needed in the definition of VTL transformations). A SDMX “Dataset”, however, is an instance of a SDMX “Dataflow” and can be the artefact on which the VTL transformations are executed (i.e., the transformations are defined on Dataflows and are applied to Dataflow instances that can be Datasets). 1103 1103 1104 -The [[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]programs (Transformation Schemes) are represented in[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] through the TransformationScheme maintainable class which is composed of Transformation ([[nameable artefact>>doc:sdmx:Glossary.Nameable artefact.WebHome]]). Each Transformation assigns the outcome of the evaluation of a[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]] expression to a result.1265 +The VTL programs (Transformation Schemes) are represented in SDMX through the TransformationScheme maintainable class which is composed of Transformation (nameable artefact). Each Transformation assigns the outcome of the evaluation of a VTL expression to a result. 1105 1105 1106 -This section does not explain the [[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]]languageor any of the content published in the[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]] guides. Rather, this is a description of how the[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]can be used in the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] context and applied to[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]].1267 +This section does not explain the VTL language or any of the content published in the VTL guides. Rather, this is a description of how the VTL can be used in the SDMX context and applied to SDMX artefacts. 1107 1107 1108 1108 == 10.2 References to SDMX artefacts from VTL statements == 1109 1109 1110 1110 === 10.2.1 Introduction === 1111 1111 1112 -The [[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]can manipulate[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]](or objects) by referencing them through pre-defined conventional names (aliases).1273 +The VTL can manipulate SDMX artefacts (or objects) by referencing them through pre-defined conventional names (aliases). 1113 1113 1114 -The alias of a [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]can be its URN (Universal Resource Name), an abbreviation of its URN or another user-defined name.1275 +The alias of a SDMX artefact can be its URN (Universal Resource Name), an abbreviation of its URN or another user-defined name. 1115 1115 1116 -In any case, the aliases used in the [[VTL>>doc:sdmx:Glossary.Validationandtransformationlanguage.WebHome]] transformationshave to be(% style="color:#e74c3c" %)mapped(%%)to the[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] through the VtlMappingScheme and VtlMapping classes (see the section of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] IM relevant to the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]]). A VtlMapping allows specifying the aliases to be used in the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] transformations, rulesets[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[5~]^^>>path:#_ftn5]](%%) or user defined operators[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[6~]^^>>path:#_ftn6]](%%) to reference [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]. A VtlMappingScheme is a container for zero or more VtlMapping.1277 +In any case, the aliases used in the VTL transformations have to be mapped to the 1117 1117 1118 - Thecorrespondencebetweenan alias and a [[SDMX>>doc:sdmx:Glossary.Statisticaldata andmetadataexchange.WebHome]][[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]mustbeone-to-one,meaningthata genericaliasidentifies oneand just one[[SDMX>>doc:sdmx:Glossary.Statisticaldata andmetadata exchange.WebHome]][[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]whilea[[SDMX>>doc:sdmx:Glossary.Statistical dataandmetadataexchange.WebHome]][[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]isidentifiedbyoneand justonealias. Inother words, withina VtlMappingSchemean[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]can have justonealias and different[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]cannothavethesamealias.1279 +SDMX artefacts through the VtlMappingScheme and VtlMapping classes (see the section of the SDMX IM relevant to the VTL). A VtlMapping allows specifying the aliases to be used in the VTL transformations, rulesets[[(% class="wikiinternallink wikiinternallink" %)^^~[5~]^^>>path:#_ftn5]](%%) or user defined operators[[(% class="wikiinternallink wikiinternallink" %)^^~[6~]^^>>path:#_ftn6]](%%) to reference SDMX artefacts. A VtlMappingScheme is a container for zero or more VtlMapping. 1119 1119 1281 +The correspondence between an alias and a SDMX artefact must be one-to-one, meaning that a generic alias identifies one and just one SDMX artefact while a SDMX artefact is identified by one and just one alias. In other words, within a VtlMappingScheme an artefact can have just one alias and different artefacts cannot have the same alias. 1282 + 1120 1120 The references through the URN and the abbreviated URN are described in the following paragraphs. 1121 1121 1122 1122 === 10.2.2 References through the URN === 1123 1123 1124 -This approach has the advantage that in the [[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]] [[code>>doc:sdmx:Glossary.Code.WebHome]]the URN of the referenced[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]is directly intelligible by a human reader but has the drawback that the references are verbose.1287 +This approach has the advantage that in the VTL code the URN of the referenced artefacts is directly intelligible by a human reader but has the drawback that the references are verbose. 1125 1125 1126 -The [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]URN[[(% class="wikiinternallink wikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[7~]^^>>path:#_ftn7]](%%) is the concatenation of the following parts, separated by special symbols like dot, equal, asterisk, comma, and parenthesis:^^ ^^1289 +The SDMX URN[[(% class="wikiinternallink wikiinternallink" %)^^~[7~]^^>>path:#_ftn7]](%%) is the concatenation of the following parts, separated by special symbols like dot, equal, asterisk, comma, and parenthesis:^^ ^^ 1127 1127 1128 -* SDMXprefix 1129 -* [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]-package-name1130 -* class-name 1131 -* agency-id 1291 +* SDMXprefix 1292 +* SDMX-IM-package-name 1293 +* class-name 1294 +* agency-id 1132 1132 * maintainedobject-id 1133 -* maintainedobject- (% style="color:#e74c3c" %)version1134 -* container-object-id [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[8~]^^>>path:#_ftn8]]1296 +* maintainedobject-version 1297 +* container-object-id [[(% class="wikiinternallink wikiinternallink" %)^^~[8~]^^>>path:#_ftn8]] 1135 1135 * object-id 1136 1136 1137 1137 The generic structure of the URN is the following: 1138 1138 1139 -SDMXprefix**.** [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]-package-name**.**class-name**=**agency-id**:**maintainedobject-id1302 +SDMXprefix**.**SDMX-IM-package-name**.**class-name**=**agency-id**:**maintainedobject-id 1140 1140 1141 -**(**maintainedobject- (% style="color:#e74c3c" %)version(%%)**).***container-object-id**.**object-id1304 +**(**maintainedobject-version**).***container-object-id**.**object-id 1142 1142 1143 -The ** [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]prefix** is “urn:[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]:org”, always the same for all[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]].1306 +The **SDMX prefix** is “urn:sdmx:org”, always the same for all SDMX artefacts. 1144 1144 1145 -The ** [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]-package-name **is the concatenation of the string** **“[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].infomodel.” with the package-name which the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]belongs to. For example, for referencing a[[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]the[[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]-package-name is “[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].infomodel.datastructure”, because the class ,,[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]],, belongs to the package “datastructure”.1308 +The **SDMX-IM-package-name **is the concatenation of the string** **“sdmx.infomodel.” with the package-name which the artefact belongs to. For example, for referencing a dataflow the SDMX-IM-package-name is “sdmx.infomodel.datastructure”, because the class ,,Dataflow,, belongs to the package “datastructure”. 1146 1146 1147 -The **class-name** is the name of the [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] object class which the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] object belongs to (e.g., for referencing a[[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]the class-name is “[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]”). The[[VTL>>doc:sdmx:Glossary.Validationand transformation language.WebHome]]can reference[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]that belong to the classes ,,[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]],[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]],,,1310 +The **class-name** is the name of the SDMX object class which the SDMX object belongs to (e.g., for referencing a dataflow the class-name is “Dataflow”). The VTL can reference SDMX artefacts that belong to the classes ,,Dataflow, Dimension,,, 1148 1148 1149 -MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, (% style="color:#e74c3c" %)Concept(%%), ConceptScheme, Codelist.1312 +MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, Concept, ConceptScheme, Codelist. 1150 1150 1151 -The **agency-id** is the acronym of the agency that owns the definition of the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]], for example for the Eurostat[[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]the agency-id is “ESTAT”). The agency-id can be composite (for example AgencyA.Dept1.Unit2).1314 +The **agency-id** is the acronym of the agency that owns the definition of the artefact, for example for the Eurostat artefacts the agency-id is “ESTAT”). The agency-id can be composite (for example AgencyA.Dept1.Unit2). 1152 1152 1153 -The **maintainedobject-id** is the name of the maintained object which the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]belongs to, and in case the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]itself is maintainable[[(% class="wikiinternallink wikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[9~]^^>>path:#_ftn9]](%%), coincides with the name of the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]].Therefore the maintainedobject-id depends on the class of the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]:1316 +The **maintainedobject-id** is the name of the maintained object which the artefact belongs to, and in case the artefact itself is maintainable[[(% class="wikiinternallink wikiinternallink" %)^^~[9~]^^>>path:#_ftn9]](%%), coincides with the name of the artefact. Therefore the maintainedobject-id depends on the class of the artefact: 1154 1154 1155 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]is a[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], which is a maintainable class, the maintainedobject-id is the[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]name ([[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]-id);1156 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]is a[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute, which are not maintainable and belong to the DataStructure maintainable class, the maintainedobject-id is the name of the DataStructure (dataStructure-id) which the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]belongs to;1157 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]is a(% style="color:#e74c3c" %)Concept(%%), which is not maintainable and belongs to the ConceptScheme maintainable class, ,, ,,the maintainedobject-id is the name of the ConceptScheme (conceptScheme-id) which the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]belongs to;1158 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]is a ConceptScheme, which is a maintainable class, ,, ,,the maintainedobject-id is the name of the ConceptScheme (conceptScheme-id);1159 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]is a Codelist,which is a maintainable class, the maintainedobject-id is the Codelist name (codelist-id).1318 +* if the artefact is a ,,Dataflow,,, which is a maintainable class, the maintainedobject-id is the Dataflow name (dataflow-id); 1319 +* if the artefact is a Dimension, MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute, which are not maintainable and belong to the ,,DataStructure,, maintainable class, the maintainedobject-id is the name of the DataStructure (dataStructure-id) which the artefact belongs to; 1320 +* if the artefact is a ,,Concept,,, which is not maintainable and belongs to the ConceptScheme maintainable class, ,, ,,the maintainedobject-id is the name of the ConceptScheme (conceptScheme-id) which the artefact belongs to; 1321 +* if the artefact is a ,,ConceptScheme,,, which is a maintainable class, ,, ,,the maintainedobject-id is the name of the ConceptScheme (conceptScheme-id); 1322 +* if the artefact is a ,,Codelist, ,,which is a maintainable class, the maintainedobject-id is the Codelist name (codelist-id). 1160 1160 1161 -The **maintainedobject- (% style="color:#e74c3c" %)version(%%)** is the(% style="color:#e74c3c" %)version(%%)of the maintained object which the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]belongs to (for example, possible(% style="color:#e74c3c" %)versions(%%)are 1.0, 2.1, 3.1.2).1324 +The **maintainedobject-version** is the version of the maintained object which the artefact belongs to (for example, possible versions are 1.0, 2.1, 3.1.2). 1162 1162 1163 -The **container-object-id** does not apply to the classes that can be referenced in [[VTL>>doc:sdmx:Glossary.Validationandtransformationlanguage.WebHome]] transformations, therefore is not present in their URN1326 +The **container-object-id** does not apply to the classes that can be referenced in VTL transformations, therefore is not present in their URN 1164 1164 1165 -The **object-id** is the name of the non- [[maintainable artefact>>doc:sdmx:Glossary.Maintainableartefact.WebHome]](when the[[artefact>>doc:sdmx:Glossary.Artefact.WebHome]]is maintainable its name is already specified as the maintainedobject-id, see above), in particular it has to be specified:1328 +The **object-id** is the name of the non-maintainable artefact (when the artefact is maintainable its name is already specified as the maintainedobject-id, see above), in particular it has to be specified: 1166 1166 1167 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] is a [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute (the object-id is the name of one of the [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] above, which are data structure [[components>>doc:sdmx:Glossary.Component.WebHome]]) 1168 -* if the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] is a (% style="color:#e74c3c" %)Concept(%%) (the object-id is the name of the (% style="color:#e74c3c" %)Concept(%%)) 1330 +* if the artefact is a Dimension, MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute (the object-id is the name of one of 1169 1169 1170 - For example, by usingtheURN, the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] transformation that sums two [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]][[dataflows>>doc:sdmx:Glossary.Dataflow.WebHome]] DF1 and DF2 and assigns theresult toathird persistent[[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]DFR, assumingthat DF1, DF2 and DFR are the maintainedobject-id of the three[[dataflows>>doc:sdmx:Glossary.Dataflow.WebHome]], that their (% style="color:#e74c3c" %)version(%%) is 1.0 and their Agency is AG, would be written as[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[10~]^^>>path:#_ftn10]](%%):1332 +the artefacts above, which are data structure components) 1171 1171 1334 +* if the artefact is a ,,Concept ,,(the object-id is the name of the ,,Concept,,) 1335 + 1336 +For example, by using the URN, the VTL transformation that sums two SDMX dataflows DF1 and DF2 and assigns the result to a third persistent dataflow DFR, assuming that DF1, DF2 and DFR are the maintainedobject-id of the three dataflows, that their version is 1.0 and their Agency is AG, would be written as[[(% class="wikiinternallink wikiinternallink" %)^^~[10~]^^>>path:#_ftn10]](%%): 1337 + 1172 1172 ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DFR(1.0)’ <- 1173 -‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF1(1.0)’ + 1339 + 1340 +‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF1(1.0)’ + 1341 + 1174 1174 ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF2(1.0)’ 1175 1175 1176 1176 === 10.2.3 Abbreviation of the URN === ... ... @@ -1180,50 +1180,52 @@ 1180 1180 The URN can be abbreviated by omitting the parts that are not essential for the identification of the artefact or that can be deduced from other available information, including the context in which the invocation is made. The possible abbreviations are described below. 1181 1181 1182 1182 * The **SDMXPrefix** can be omitted for all the SDMX objects, because it is a prefixed string (urn:sdmx:org), always the same for SDMX objects. 1183 -* The **SDMX-IM-package-name **can be omitted as well because it can be deduced from the class-name that follows it (the table of the SDMX-IM packages and classes that allows this deduction is in the SDMX 2.1 Standards - Section 5 - Registry Specifications, paragraph 6.2.3). In particular, considering the object classes of the artefacts that VTL can reference, the package is: 1184 -** “datastructure” for the classes Dataflow, Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, 1185 -** “conceptscheme” for the classes Concept and ConceptScheme 1186 -** “codelist” for the class Codelist. 1187 -* The **class-name** can be omitted as it can be deduced from the VTL invocation. In particular, starting from the VTL class of the invoked artefact (e.g. dataset, component, identifier, measure, attribute, variable, valuedomain), which is known given the syntax of the invoking VTL operator[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[11~]^^>>path:#_ftn11]](%%), the SDMX class can be deduced from the mapping rules between VTL and SDMX (see the section “Mapping between VTL and SDMX” hereinafter)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[12~]^^>>path:#_ftn12]](%%). 1188 -* If the **agency-id** is not specified, it is assumed by default equal to the agency-id of the TransformationScheme, UserDefinedOperatorScheme or RulesetScheme from which the artefact is invoked. For example, the agency-id can be omitted if it is the same as the invoking TransformationScheme and cannot be omitted if the artefact comes from another agency.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[13~]^^>>path:#_ftn13]](%%) Take also into account that, according to the VTL consistency rules, the agency of the result of a Transformation must be the same as its TransformationScheme, therefore the agency-id can be omitted for all the results (left part of Transformation statements). 1189 -* As for the **maintainedobject-id**, this is essential in some cases while in other cases it can be omitted: o if the referenced artefact is a Dataflow, which is a maintainable class, the maintainedobject-id is the dataflow-id and obviously cannot be omitted; 1190 -** if the referenced artefact is a Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, which are not maintainable and belong to the DataStructure maintainable class, the maintainedobject-id is the dataStructure-id and can be omitted, given that these components are always invoked within the invocation of a Dataflow, whose dataStructure-id can be deduced from the SDMX structural definitions; 1191 -** if the referenced artefact is a Concept, which is not maintainable and belong to the ConceptScheme maintainable class,,, ,,the maintained object is the conceptScheme-id and cannot be omitted; 1192 -** if the referenced artefact is a ConceptScheme, which is a,, ,,maintainable class,,, ,,the maintained object is the conceptScheme-id and obviously cannot be omitted; 1193 -** if the referenced artefact is a Codelist, which is a maintainable class, the maintainedobject-id is the codelist-id and obviously cannot be omitted. 1351 +* The **SDMX-IM-package-name **can be omitted as well because it can be deduced from the class-name that follows it (the table of the SDMX-IM packages and classes that allows this deduction is in the SDMX 2.1 Standards - Section 5 - Registry Specifications, paragraph 6.2.3). In particular, considering the object classes of the artefacts that VTL can reference, the package is: 1352 +** “datastructure” for the classes Dataflow, Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, 1353 +** “conceptscheme” for the classes Concept and ConceptScheme o “codelist” for the class Codelist. 1354 +* The **class-name** can be omitted as it can be deduced from the VTL invocation. In particular, starting from the VTL class of the invoked artefact (e.g. dataset, component, identifier, measure, attribute, variable, valuedomain), which is known given the syntax of the invoking VTL operator[[(% class="wikiinternallink wikiinternallink" %)^^~[11~]^^>>path:#_ftn11]](%%), the SDMX class can be deduced from the mapping rules between VTL and SDMX (see the section “Mapping between VTL and SDMX” hereinafter)[[(% class="wikiinternallink wikiinternallink" %)^^~[12~]^^>>path:#_ftn12]](%%). 1355 +* If the **agency-id** is not specified, it is assumed by default equal to the agency-id of the TransformationScheme, UserDefinedOperatorScheme or RulesetScheme from which the artefact is invoked. For example, the agency-id can be omitted if it is the same as the invoking T,,ransformationScheme,, and cannot be omitted if the artefact comes from another agency.[[(% class="wikiinternallink wikiinternallink" %)^^~[13~]^^>>path:#_ftn13]](%%) Take also into account that, according to the VTL consistency rules, the agency of the result of a ,,Transformation,, must be the same as its ,,TransformationScheme,,, therefore the agency-id can be omitted for all the results (left part of ,,Transformation,, statements). 1356 +* As for the **maintainedobject-id**, this is essential in some cases while in other cases it can be omitted: o if the referenced artefact is a ,,Dataflow,,, which is a maintainable class, the maintainedobject-id is the dataflow-id and obviously cannot be omitted; 1357 +** if the referenced artefact is a Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, which are not maintainable and belong to the ,,DataStructure,, maintainable class, the maintainedobject-id is the dataStructure-id and can be omitted, given that these components are always invoked within the invocation of a ,,Dataflow,,, whose dataStructure-id can be deduced from the 1358 + 1359 +SDMX structural definitions; o if the referenced artefact is a ,,Concept, ,,which is not maintainable and belong to the ,,ConceptScheme ,,maintainable class,,, ,,the maintained object is the conceptScheme-id and cannot be omitted; 1360 + 1361 +* 1362 +** if the referenced artefact is a ,,ConceptScheme, ,,which is a,, ,,maintainable class,,, ,,the maintained object is the ,,conceptScheme-id,, and obviously cannot be omitted; 1363 +** if the referenced artefact is a ,,Codelist, ,,which is a maintainable class, the maintainedobject-id is the ,,codelist-id,, and obviously cannot be omitted. 1194 1194 * When the maintainedobject-id is omitted, the **maintainedobject-version** is omitted too. When the maintainedobject-id is not omitted and the maintainedobject-version is omitted, the version 1.0 is assumed by default.,, ,, 1195 1195 * As said, the **container-object-id** does not apply to the classes that can be referenced in VTL transformations, therefore is not present in their URN 1196 -* The **object-id** does not exist for the artefacts belonging to the Dataflow, ConceptScheme and Codelist classes, while it exists and cannot be omitted for the artefacts belonging to the classes Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute and Concept, as for them the object-id is the main identifier of the artefact1366 +* The **object-id** does not exist for the artefacts belonging to the ,,Dataflow, ConceptScheme,, and ,,Codelist,, classes, while it exists and cannot be omitted for the artefacts belonging to the classes Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute and Concept, as for 1197 1197 1368 +them the object-id is the main identifier of the artefact 1369 + 1198 1198 The simplified object identifier is obtained by omitting all the first part of the URN, including the special characters, till the first part not omitted. 1199 1199 1200 1200 For example, the full formulation that uses the complete URN shown at the end of the previous paragraph: 1201 1201 1202 -‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DFR(1.0)’ := 1203 - ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF1(1.0)’ +1374 +‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DFR(1.0)’ := ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF1(1.0)’ + 1375 + 1204 1204 ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF2(1.0)’ 1205 1205 1206 -by omitting all the non-essential parts would become simply: 1378 +by omitting all the non-essential parts would become simply: 1207 1207 1208 -DFR := DF1 + DF2 1380 +DFR := DF1 + DF2 1209 1209 1210 -The references to the Codelists can be simplified similarly. For example, given the non-abbreviated reference to the Codelist AG:CL_FREQ(1.0), which is[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[14~]^^>>path:#_ftn14]](%%):1382 +The references to the ,,Codelists,, can be simplified similarly. For example, given the non-abbreviated reference to the ,,Codelist,, AG:CL_FREQ(1.0), which is[[(% class="wikiinternallink wikiinternallink" %)^^~[14~]^^>>path:#_ftn14]](%%): 1211 1211 1212 1212 ‘urn:sdmx:org.sdmx.infomodel.codelist.Codelist=AG:CL_FREQ(1.0)’ 1213 1213 1214 -if the Codelist is referenced from a ruleset scheme belonging to the agency AG, omitting all the optional parts, the abbreviated reference would become simply[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[15~]^^>>path:#_ftn15]](%%):1386 +if the ,,Codelist,, is referenced from a ruleset scheme belonging to the agency AG, omitting all the optional parts, the abbreviated reference would become simply[[(% class="wikiinternallink wikiinternallink" %)^^~[15~]^^>>path:#_ftn15]](%%): 1215 1215 1216 1216 CL_FREQ 1217 1217 1218 -As for the references to the components, it can be enough to specify the componentId, given that the dataStructure-Id can be omitted. An example of non-abbreviated reference, if the data structure is DST1 and the component is SECTOR, is the following: 1390 +As for the references to the components, it can be enough to specify the componentId, given that the dataStructure-Id can be omitted. An example of non-abbreviated reference, if the data structure is DST1 and the component is SECTOR, is the following: 1219 1219 1220 -‘urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=AG:DST1(1.0).SECTOR’ 1392 +‘urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=AG:DST1(1.0).SECTOR’ The corresponding fully abbreviated reference, if made from a transformation scheme belonging to AG, would become simply: 1221 1221 1222 -The corresponding fully abbreviated reference, if made from a transformation scheme belonging to AG, would become simply: 1223 - 1224 1224 SECTOR 1225 1225 1226 -For example, the transformation for renaming the component SECTOR of the dataflow DF1 into SEC can be written as[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[16~]^^>>path:#_ftn16]](%%):1396 +For example, the transformation for renaming the component SECTOR of the dataflow DF1 into SEC can be written as[[(% class="wikiinternallink wikiinternallink" %)^^~[16~]^^>>path:#_ftn16]](%%): 1227 1227 1228 1228 ‘DFR(1.0)’ := ‘DF1(1.0)’ [rename SECTOR to SEC] 1229 1229 ... ... @@ -1233,7 +1233,7 @@ 1233 1233 1234 1234 ‘urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=AG:CS1(1.0).SECTOR’ 1235 1235 1236 -The corresponding fully abbreviated reference, if made from a RulesetScheme belonging to AG, would become simply: 1406 +The corresponding fully abbreviated reference, if made from a RulesetScheme belonging to AG, would become simply: 1237 1237 1238 1238 CS1(1.0).SECTOR 1239 1239 ... ... @@ -1255,13 +1255,13 @@ 1255 1255 1256 1256 VTL operators, like the ones for validation and hierarchical roll-up. A “rule” consists in a relationship between Values belonging to some Value Domains or taken by some Variables, for example: (i) when the Country is USA then the Currency is USD; (ii) the Benelux is composed by Belgium, Luxembourg, Netherlands. 1257 1257 1258 -The VTL Rulesets have a signature, in which the Value Domains or the Variables on which the Ruleset is defined are declared, and a body, which contains the rules. 1428 +The VTL Rulesets have a signature, in which the Value Domains or the Variables on which the Ruleset is defined are declared, and a body, which contains the rules. 1259 1259 1260 -In the signature, given the mapping between VTL and SDMX better described in the following paragraphs, a reference to a VTL Value Domain becomes a reference to a SDMX Codelist or to a SDMX ConceptScheme (for SDMX measure dimensions), while a reference to a VTL Represented Variable becomes a reference to a SDMX Concept, assuming for it a definite representation[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[17~]^^>>path:#_ftn17]](%%).1430 +In the signature, given the mapping between VTL and SDMX better described in the following paragraphs, a reference to a VTL Value Domain becomes a reference to a SDMX Codelist or to a SDMX ConceptScheme (for SDMX measure dimensions), while a reference to a VTL Represented Variable becomes a reference to a SDMX Concept, assuming for it a definite representation[[(% class="wikiinternallink wikiinternallink" %)^^~[17~]^^>>path:#_ftn17]](%%). 1261 1261 1262 -In general, for referencing SDMX Codelists and Concepts, the conventions described in the previous paragraphs apply. In the Ruleset syntax, the elements that reference SDMX artefacts are called “valueDomain” and “variable” for the Datapoint Rulesets and “ruleValueDomain”, “ruleVariable”, “condValueDomain” “condVariable” for the Hierarchical Rulesets). The syntax of the Ruleset signature allows also to define aliases of the elements above, these aliases are valid only within the specific ruleset definition statement and cannot be mapped to SDMX.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[18~]^^>>path:#_ftn18]](%%)1432 +In general, for referencing SDMX Codelists and Concepts, the conventions described in the previous paragraphs apply. In the Ruleset syntax, the elements that reference SDMX artefacts are called “valueDomain” and “variable” for the Datapoint Rulesets and “ruleValueDomain”, “ruleVariable”, “condValueDomain” “condVariable” for the Hierarchical Rulesets). The syntax of the Ruleset signature allows also to define aliases of the elements above, these aliases are valid only within the specific ruleset definition statement and cannot be mapped to SDMX.[[(% class="wikiinternallink wikiinternallink" %)^^~[18~]^^>>path:#_ftn18]](%%) 1263 1263 1264 -In the body of the Rulesets, the Codes and in general all the Values can be written without any other specification, because the artefact which the Values are referred (Codelist, ConceptScheme, Concept) to can be deduced from the Ruleset signature. 1434 +In the body of the Rulesets, the Codes and in general all the Values can be written without any other specification, because the artefact which the Values are referred (Codelist, ConceptScheme, Concept) to can be deduced from the Ruleset signature. 1265 1265 1266 1266 == 10.3 Mapping between SDMX and VTL artefacts == 1267 1267 ... ... @@ -1269,59 +1269,62 @@ 1269 1269 1270 1270 The mapping methods between the VTL and SDMX object classes allow transforming a SDMX definition in a VTL one and vice-versa for the artefacts to be manipulated. 1271 1271 1272 -It should be remembered that VTL programs (i.e. Transformation Schemes) are represented in SDMX through the TransformationScheme maintainable class which is composed of Transformations (nameable artefacts). Each Transformation assigns the outcome of the evaluation of a VTL expression to a result: the input operands of the expression and the result can be SDMX artefacts. 1442 +It should be remembered that VTL programs (i.e. Transformation Schemes) are represented in SDMX through the TransformationScheme maintainable class which is composed of Transformations (nameable artefacts). Each Transformation assigns the outcome of the evaluation of a VTL expression to a result: the input operands of the expression and the result can be SDMX artefacts. 1273 1273 1274 -Every time a SDMX object is referenced in a VTL Transformation as an input operand, there is the need to generate a VTL definition of the object, so that the VTL operations can take place. This can be made starting from the SDMX definition and applying a SDMX-VTL mapping method in the direction from SDMX to VTL. The possible mapping methods from SDMX to VTL are described in the following paragraphs and are conceived to allow the automatic deduction of the VTL definition of the object from the knowledge of the SDMX definition. 1444 +Every time a SDMX object is referenced in a VTL Transformation as an input operand, there is the need to generate a VTL definition of the object, so that the VTL operations can take place. This can be made starting from the SDMX definition and applying a SDMX-VTL mapping method in the direction from SDMX to VTL. The possible mapping methods from SDMX to VTL are described in the following paragraphs and are conceived to allow the automatic deduction of the VTL definition of the object from the knowledge of the SDMX definition. 1275 1275 1276 -In the opposite direction, every time an object calculated by means of VTL must be treated as a SDMX object (for example for exchanging it through SDMX), there is the need of a SDMX definition of the object, so that the SDMX operations can take place. The SDMX definition is needed for the VTL objects for which a SDMX use is envisaged[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[19~]^^>>path:#_ftn19]](%%).1446 +In the opposite direction, every time an object calculated by means of VTL must be treated as a SDMX object (for example for exchanging it through SDMX), there is the need of a SDMX definition of the object, so that the SDMX operations can take place. The SDMX definition is needed for the VTL objects for which a SDMX use is envisaged[[(% class="wikiinternallink wikiinternallink" %)^^~[19~]^^>>path:#_ftn19]](%%). 1277 1277 1278 -The mapping methods from VTL to SDMX are described in the following paragraphs as well, however they do not allow the complete SDMX definition to be automatically deduced from the VTL definition, more than all because the former typically contains additional information in respect to the latter. For example, the definition of a SDMX DSD includes also some mandatory information not available in VTL (like the concept scheme to which the SDMX components refer, the assignmentStatus and attributeRelationship for the DataAttributes and so on). Therefore the mapping methods from VTL to SDMX provide only a general guidance for generating SDMX definitions properly starting from the information available in VTL, independently of how the SDMX definition it is actually generated (manually, automatically or part and part). 1448 +The mapping methods from VTL to SDMX are described in the following paragraphs as well, however they do not allow the complete SDMX definition to be automatically deduced from the VTL definition, more than all because the former typically contains additional information in respect to the latter. For example, the definition of a SDMX DSD includes also some mandatory information not available in VTL (like the concept scheme to which the SDMX components refer, the assignmentStatus and attributeRelationship for the DataAttributes and so on). Therefore the mapping methods from VTL to SDMX provide only a general guidance for generating SDMX definitions properly starting from the information available in VTL, independently of how the SDMX definition it is actually generated (manually, automatically or part and part). 1279 1279 1280 1280 === 10.3.2 General mapping of VTL and SDMX data structures === 1281 1281 1282 -This section makes reference to the VTL “Model for data and their structure”[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[20~]^^>>path:#_ftn20]](%%) and the correspondent SDMX “Data Structure Definition”[[(% class="wikiinternallink wikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[21~]^^>>path:#_ftn21]](%%).1452 +This section makes reference to the VTL “Model for data and their structure”[[(% class="wikiinternallink wikiinternallink" %)^^~[20~]^^>>path:#_ftn20]](%%) and the correspondent SDMX “Data Structure Definition”[[(% class="wikiinternallink wikiinternallink" %)^^~[21~]^^>>path:#_ftn21]](%%). 1283 1283 1284 -The main type of artefact that the VTL can manipulate is the VTL Data Set, which in general is mapped to the SDMX Dataflow. This means that a VTL Transformation, in the SDMX context, expresses the algorithm for calculating a derived Dataflow starting from some already existing Dataflows (either collected or derived).[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[22~]^^>>path:#_ftn22]](%%)1454 +The main type of artefact that the VTL can manipulate is the VTL Data Set, which in general is mapped to the SDMX Dataflow. This means that a VTL Transformation, in the SDMX context, expresses the algorithm for calculating a derived Dataflow starting from some already existing Dataflows (either collected or derived).[[(% class="wikiinternallink wikiinternallink" %)^^~[22~]^^>>path:#_ftn22]](%%) 1285 1285 1286 -While the VTL Transformations are defined in term of Dataflow definitions, they are assumed to be executed on instances of such Dataflows, provided at runtime to the VTL engine (the mechanism for identifying the instances to be processed are not part of the VTL specifications and depend on the implementation of the VTL-based systems). As already said, the SDMX Datasets are instances of SDMX Dataflows, therefore a VTL Transformation defined on some SDMX Dataflows can be applied on some corresponding SDMX Datasets. 1456 +While the VTL Transformations are defined in term of Dataflow definitions, they are assumed to be executed on instances of such Dataflows, provided at runtime to the VTL engine (the mechanism for identifying the instances to be processed are not part of the VTL specifications and depend on the implementation of the VTL-based systems). As already said, the SDMX Datasets are instances of SDMX Dataflows, therefore a VTL Transformation defined on some SDMX Dataflows can be applied on some corresponding SDMX Datasets. 1287 1287 1288 1288 A VTL Data Set is structured by one and just one Data Structure and a VTL Data Structure can structure any number of Data Sets. Correspondingly, in the SDMX context a SDMX Dataflow is structured by one and just one DataStructureDefinition and one DataStructureDefinition can structure any number of Dataflows. 1289 1289 1290 -A VTL Data Set has a Data Structure made of Components, which in turn can be Identifiers, Measures and Attributes. Similarly, a SDMX DataflowDefinition has a DataStructureDefinition made of components that can be DimensionComponents, PrimaryMeasure and DataAttributes. In turn, a SDMX DimensionComponent can be a Dimension, a TimeDimension or a MeasureDimension. Correspondingly, in the SDMX implementation of the VTL, the VTL Identifiers can be (optionally) distinguished in three sub-classes (Simple Identifier, Time Identifier, Measure Identifier) even if such a distinction is not evidenced in the VTL IM.1460 +A VTL Data Set has a Data Structure made of Components, which in turn can be Identifiers, Measures and Attributes. Similarly, a SDMX DataflowDefinition has a DataStructureDefinition made of components that can be DimensionComponents, PrimaryMeasure and DataAttributes. In turn, a 1291 1291 1292 - However, a VTLDataStructurecanhave any numberof Identifiers,Measures andAttributes,whileaSDMX 2.1 DataStructureDefinitioncanhave any number ofDimensionsand DataAttributes but justone PrimaryMeasure[[(% class="wikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallink wikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallink wikiinternallinkwikiinternallink" %)^^~[23~]^^>>path:#_ftn23]](%%). Thisis dueto adifferencebetweenSDMX2.1andVTLin the possible representationmethods ofthedata that containmoremeasures.1462 +SDMX DimensionComponent can be a Dimension, a TimeDimension or a MeasureDimension. Correspondingly, in the SDMX implementation of the VTL, the VTL Identifiers can be (optionally) distinguished in three sub-classes (Simple Identifier, Time Identifier, Measure Identifier) even if such a distinction is not evidenced in the VTL IM. 1293 1293 1294 - As forSDMX,becausethedatastructure cannotcontain morethanonemeasurecomponent(i.e.,the primaryMeasure),the representationof datahaving moremeasuresis possible onlybymeansofaparticulardimension, calledMeasureDimension,whichis aimed at containingthe name ofthemeasureconcepts, sothatforeachobservationthevaluecontained in thePrimaryMeasurecomponentisthevalueof themeasureconceptreportedin theMeasureDimension component.1464 +However, a VTL Data Structure can have any number of Identifiers, Measures and Attributes, while a SDMX 2.1 DataStructureDefinition can have any number of Dimensions and DataAttributes but just one PrimaryMeasure[[(% class="wikiinternallink wikiinternallink" %)^^~[23~]^^>>path:#_ftn23]](%%). This is due to a difference between SDMX 2.1 and VTL in the possible representation methods of the data that contain more measures. 1295 1295 1296 - InsteadVTLallowseitherthemethodabove(anidentifier containingthe nameof the measuretogether withjust one measure component)oramoregenericmethodthatconsistsindefiningmore measure componentsinthedatastructure,oneforeachmeasure.1466 +As for SDMX, because the data structure cannot contain more than one measure component (i.e., the primaryMeasure), the representation of data having more measures is possible only by means of a particular dimension, called MeasureDimension, which is aimed at containing the name of the measure concepts, so that for each observation the value contained in the PrimaryMeasure component is the value of the measure concept reported in the MeasureDimension component. 1297 1297 1468 +Instead VTL allows either the method above (an identifier containing the name of the measure together with just one measure component) or a more generic method that consists in defining more measure components in the data structure, one for each measure. 1469 + 1298 1298 Therefore for multi-measure data more mapping options are possible, as described in more detail in the following sections. 1299 1299 1300 1300 === 10.3.3 Mapping from SDMX to VTL data structures === 1301 1301 1302 - ====10.3.3.1 Basic Mapping****====1474 +**10.3.3.1 Basic Mapping ** 1303 1303 1304 -The main mapping method from SDMX to VTL is called **Basic **mapping. This is considered as the default mapping method and is applied unless a different method is specified through the VtlMappingScheme and VtlDataflowMapping classes. 1476 +The main mapping method from SDMX to VTL is called **Basic **mapping. This is considered as the default mapping method and is applied unless a different method is specified through the VtlMappingScheme and VtlDataflowMapping classes. 1842 When transforming **from SDMX to VTL**, this method consists in leaving the 1843 components unchanged and maintaining their names and roles, according to the 1844 following table: 1305 1305 1306 -When transforming **from SDMX to VTL**, this method consists in leaving the components unchanged and maintaining their names and roles, according to the following table: 1478 +|SDMX|VTL 1479 +|Dimension|(Simple) Identifier 1480 +|Time Dimension|(Time) Identifier 1481 +|Measure Dimension|(Measure) Identifier 1482 +|Primary Measure|Measure 1483 +|Data Attribute|Attribute 1307 1307 1308 -(% style="width:636.294px" %) 1309 -|(% style="width:286px" %)**SDMX**|(% style="width:347px" %)**VTL** 1310 -|(% style="width:286px" %)Dimension|(% style="width:347px" %)(Simple) Identifier 1311 -|(% style="width:286px" %)Time Dimension|(% style="width:347px" %)(Time) Identifier 1312 -|(% style="width:286px" %)Measure Dimension|(% style="width:347px" %)(Measure) Identifier 1313 -|(% style="width:286px" %)Primary Measure|(% style="width:347px" %)Measure 1314 -|(% style="width:286px" %)Data Attribute|(% style="width:347px" %)Attribute 1485 +According to this method, the resulting VTL structures are always mono-measure 1315 1315 1316 - According to this method, the resulting VTL structures are always mono-measure(i.e., they have just one measure component) and their Measure is the SDMXPrimaryMeasure. Nevertheless, if the SDMX data structure has a MeasureDimension, which can convey the name of one or more measure concepts, such unique measure component can contain the value of more (conceptual) measures (one for each observation).1487 +(i.e., they have just one measure component) and their Measure is the SDMX 1317 1317 1489 +PrimaryMeasure. Nevertheless, if the SDMX data structure has a MeasureDimension, which can convey the name of one or more measure concepts, such unique measure component can contain the value of more (conceptual) measures (one for each observation). 1490 + 1318 1318 As for the SDMX DataAttributes, in VTL they are all considered “at data point / observation level” (i.e. dependent on all the VTL Identifiers), because VTL does not have the SDMX AttributeRelationships, which defines the construct to which the DataAttribute is related (e.g. observation, dimension or set or group of dimensions, whole data set). 1319 1319 1320 1320 With the Basic mapping, one SDMX observation generates one VTL data point. 1321 1321 1322 - ====10.3.3.2 Pivot Mapping====1495 +**10.3.3.2 Pivot Mapping ** 1323 1323 1324 -An alternative mapping method from SDMX to VTL is the **Pivot **mapping, which is different from the Basic method only for the SDMX data structures that contain a MeasureDimension, which are mapped to multi-measure VTL data structures. 1497 +An alternative mapping method from SDMX to VTL is the **Pivot **mapping, which is different from the Basic method only for the SDMX data structures that contain a MeasureDimension, which are mapped to multi-measure VTL data structures. 1325 1325 1326 1326 The SDMX structures that do not contain a MeasureDimension are mapped like in the Basic mapping (see the previous paragraph). 1327 1327 ... ... @@ -1332,34 +1332,36 @@ 1332 1332 * The SDMX MeasureDimension is not mapped to VTL (it disappears in the VTL Data Structure); 1333 1333 * The SDMX PrimaryMeasure is not mapped to VTL as well (it disappears in the VTL Data Structure); 1334 1334 * A SDMX DataAttribute is mapped in different ways according to its AttributeRelationship: 1335 -** If, according to the SDMX AttributeRelationship, the values of the DataAttribute do not depend on the values of the MeasureDimension, the SDMX DataAttribute becomes a VTL Attribute having the same name. This happens if the AttributeRelationship is not specified (i.e. the DataAttribute does not depend on any DimensionComponent and therefore is at data set level), or if it refers to a set (or a group) of dimensions which does not include the MeasureDimension; 1336 -** Otherwise if, according to the SDMX AttributeRelationship, the values of the DataAttribute depend on the MeasureDimension, the SDMX DataAttribute is mapped to one VTL Attribute for each possible Concept of the SDMX MeasureDimension; by default, the names of the VTL Attributes are obtained by concatenating the name of the SDMX DataAttribute and the names of the correspondent 1508 +** If, according to the SDMX AttributeRelationship, the values of the DataAttribute do not depend on the values of the MeasureDimension, the SDMX DataAttribute becomes a VTL Attribute having the same name. This happens if the AttributeRelationship is not specified (i.e. the DataAttribute does not depend on any DimensionComponent and therefore is at data set level), or if it refers to a set (or a group) of dimensions which does not include the MeasureDimension; 1509 +** Otherwise if, according to the SDMX AttributeRelationship, the values of the DataAttribute depend on the MeasureDimension, the SDMX DataAttribute is mapped to one VTL Attribute for each possible Concept of the SDMX MeasureDimension; by default, the names of the VTL Attributes are obtained by concatenating the name of the SDMX DataAttribute and the names of the correspondent 1337 1337 1338 1338 Concept of the MeasureDimension separated by underscore; for example, if the SDMX DataAttribute is named DA and the possible concepts of the SDMX MeasureDimension are named C1, C2, …, Cn, then the corresponding VTL Attributes will be named DA_C1, DA_C2, …, DA_Cn (if different names are desired, they can be achieved afterwards by renaming the Attributes through VTL operators). o Like in the Basic mapping, the resulting VTL Attributes are considered as dependent on all the VTL identifiers (i.e. “at data point / observation level”), because VTL does not have the SDMX notion of Attribute Relationship. 1339 1339 1340 1340 The summary mapping table of the “pivot” mapping from SDMX to VTL for the SDMX data structures that contain a MeasureDimension is the following: 1341 1341 1342 -(% style="width:941.294px" %) 1343 -|(% style="width:441px" %)**SDMX**|(% style="width:497px" %)**VTL** 1344 -|(% style="width:441px" %)Dimension|(% style="width:497px" %)(Simple) Identifier 1345 -|(% style="width:441px" %)TimeDimension|(% style="width:497px" %)(Time) Identifier 1346 -|(% style="width:441px" %)MeasureDimension & PrimaryMeasure|(% style="width:497px" %)One Measure for each Concept of the SDMX Measure Dimension 1347 -|(% style="width:441px" %)DataAttribute not depending on the MeasureDimension|(% style="width:497px" %)Attribute 1348 -|(% style="width:441px" %)DataAttribute depending on the MeasureDimension|(% style="width:497px" %)One Attribute for each Concept of the SDMX Measure Dimension 1515 +|SDMX|VTL 1516 +|Dimension|(Simple) Identifier 1517 +|TimeDimension|(Time) Identifier 1518 +|MeasureDimension & PrimaryMeasure|One Measure for each Concept of the SDMX Measure Dimension 1519 +|DataAttribute not depending on the MeasureDimension|Attribute 1520 +|DataAttribute depending on the MeasureDimension|One Attribute for each Concept of the SDMX Measure Dimension 1349 1349 1350 -Using this mapping method, the components of the data structure can change in the conversion from SDMX to VTL and it must be taken into account that the VTL statements can reference only the components of the resulting VTL data structure. 1522 +Using this mapping method, the components of the data structure can change in the conversion from SDMX to VTL and it must be taken into account that the VTL 1908 statements can reference only the components of the resulting VTL data structure. 1351 1351 1352 -At observation / data point level, calling Cj (j=1, … n) the j^^th^^ Concept of the MeasureDimension: 1524 +At observation / data point level, calling Cj (j=1, … n) the j^^th^^ Concept of the 1911 MeasureDimension: 1353 1353 1354 -* The set of SDMX observations having the same values for all the Dimensions except than the MeasureDimension become one multi-measure VTL Data Point, having one Measure for each Concept Cj of the SDMX MeasureDimension; 1355 -* The values of the SDMX simple Dimensions, TimeDimension and DataAttributes not depending on the MeasureDimension (these components by definition have always the same values for all the observations of the set above) become the values of the corresponding VTL (simple) Identifiers, (time) Identifier and Attributes. 1356 -* The value of the PrimaryMeasure of the SDMX observation belonging to the set above and having MeasureDimension=Cj becomes the value of the VTL Measure Cj 1357 -* For the SDMX DataAttributes depending on the MeasureDimension, the value of the DataAttribute DA of the SDMX observation belonging to the set above and having MeasureDimension=Cj becomes the value of the VTL Attribute DA_Cj 1526 + The set of SDMX observations having the same values for all the Dimensions except than the MeasureDimension become one multi-measure VTL Data Point, having one Measure for each Concept Cj of the SDMX MeasureDimension; 1358 1358 1359 -==== 10.3.3.3 From SDMX DataAttributes to VTL Measures ==== 1528 +* 1529 +** The values of the SDMX simple Dimensions, TimeDimension and DataAttributes not depending on the MeasureDimension (these components by definition have always the same values for all the observations of the set above) become the values of the corresponding VTL (simple) Identifiers, (time) Identifier and Attributes. 1530 +** The value of the PrimaryMeasure of the SDMX observation belonging to the set above and having MeasureDimension=Cj becomes the value of the VTL Measure Cj 1531 +** For the SDMX DataAttributes depending on the MeasureDimension, the value of the DataAttribute DA of the SDMX observation belonging to the set above and having MeasureDimension=Cj becomes the value of the VTL Attribute DA_Cj 1360 1360 1361 -* In some cases it may happen that the DataAttributes of the SDMX DataStructure need to be managed as Measures in VTL.Therefore, a variant of both themethodsabove consists in transforming all theSDMX DataAttributesin VTL Measures. When DataAttributes are convertedto Measures, the two methods above are called Basic_A2M and Pivot_A2M (the suffix “A2M” stands for Attributes to Measures). Obviously, the resultingVTLdata structureis, in general, multi-measureand doesnot contain Attributes.1533 +**10.3.3.3 From SDMX DataAttributes to VTL Measures ** 1362 1362 1535 +* 1536 +** In some cases it may happen that the DataAttributes of the SDMX DataStructure need to be managed as Measures in VTL. Therefore, a variant of both the methods above consists in transforming all the SDMX DataAttributes in VTL Measures. When DataAttributes are converted to Measures, the two methods above are called Basic_A2M and Pivot_A2M (the suffix “A2M” stands for Attributes to Measures). Obviously, the resulting VTL data structure is, in general, multi-measure and does not contain Attributes. 1537 + 1363 1363 The Basic_A2M and Pivot_A2M behaves respectively like the Basic and Pivot methods, except that the final VTL components, which according to the Basic and Pivot methods would have had the role of Attribute, assume instead the role of Measure. 1364 1364 1365 1365 Proper VTL features allow changing the role of specific attributes even after the SDMX to VTL mapping: they can be useful when only some of the DataAttributes need to be managed as VTL Measures. ... ... @@ -1366,27 +1366,28 @@ 1366 1366 1367 1367 === 10.3.4 Mapping from VTL to SDMX data structures === 1368 1368 1369 - ====10.3.4.1 Basic Mapping****====1544 +**10.3.4.1 Basic Mapping ** 1370 1370 1371 1371 The main mapping method **from VTL to SDMX** is called **Basic **mapping as well. 1372 1372 1373 -This is considered as the default mapping method and is applied unless a different method is specified through the VtlMappingScheme and VtlDataflowMapping classes. 1548 +This is considered as the default mapping method and is applied unless a different method is specified through the VtlMappingScheme and VtlDataflowMapping classes. 1374 1374 1375 1375 The method consists in leaving the components unchanged and maintaining their names and roles in SDMX, according to the following mapping table, which is the same as the basic mapping from SDMX to VTL, only seen in the opposite direction. 1376 1376 1377 -This mapping method cannot be applied for SDMX 2.1 if the VTL data structure has more than one measure component, given that the SDMX 2.1 DataStructureDefinition allows just one measure component (the PrimaryMeasure). In this case it becomes mandatory to specify a different mapping method through the VtlMappingScheme and VtlDataflowMapping classes.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[24~]^^>>path:#_ftn24]](%%)1552 +This mapping method cannot be applied for SDMX 2.1 if the VTL data structure has more than one measure component, given that the SDMX 2.1 DataStructureDefinition allows just one measure component (the 1378 1378 1379 -P lease notethattheVTLmeasurescanhave anynamewhileinSDMX2.1 theMeasureComponenthasthemandatoryname“obs_value”,thereforethenameoftheVTLmeasurename mustbecome “obs_value”inSDMX2.1.1554 +PrimaryMeasure). In this case it becomes mandatory to specify a different 1958 mapping method through the VtlMappingScheme and VtlDataflowMapping 1959 classes.[[(% class="wikiinternallink wikiinternallink" %)^^~[24~]^^>>path:#_ftn24]](%%) 1380 1380 1556 +1960 Please note that the VTL measures can have any name while in SDMX 2.1 the 1961 MeasureComponent has the mandatory name “obs_value”, therefore the name of the VTL measure name must become “obs_value” in SDMX 2.1. 1557 + 1381 1381 Mapping table: 1382 1382 1383 -(% style="width:592.294px" %) 1384 -|(% style="width:253px" %)**VTL**|(% style="width:336px" %)**SDMX** 1385 -|(% style="width:253px" %)(Simple) Identifier|(% style="width:336px" %)Dimension 1386 -|(% style="width:253px" %)(Time) Identifier|(% style="width:336px" %)TimeDimension 1387 -|(% style="width:253px" %)(Measure) Identifier|(% style="width:336px" %)MeasureDimension 1388 -|(% style="width:253px" %)Measure|(% style="width:336px" %)PrimaryMeasure 1389 -|(% style="width:253px" %)Attribute|(% style="width:336px" %)DataAttribute 1560 +|VTL|SDMX 1561 +|(Simple) Identifier|Dimension 1562 +|(Time) Identifier|TimeDimension 1563 +|(Measure) Identifier|MeasureDimension 1564 +|Measure|PrimaryMeasure 1565 +|Attribute|DataAttribute 1390 1390 1391 1391 If the distinction between simple identifier, time identifier and measure identifier is not maintained in the VTL environment, the classification between Dimension, TimeDimension and MeasureDimension exists only in SDMX, as declared in the relevant DataStructureDefinition. 1392 1392 ... ... @@ -1394,14 +1394,16 @@ 1394 1394 1395 1395 Note that the basic mappings in the two directions (from SDMX 2.1 to VTL 2.0 and vice-versa) are (almost completely) reversible. In fact, if a SDMX 2.1 structure is mapped to a VTL structure and then the latter is mapped back to SDMX 2.1, the resulting data structure is like the original one (apart for the AttributeRelationship, that can be different if the original SDMX 2.1 structure contains DataAttributes that are not at observation level). In reverse order, if a VTL 2.0 mono-measure structure is mapped to SDMX 2.1 and then the latter is mapped back to VTL 2.0, the original data structure is obtained (apart from the name of the VTL measure, that in SDMX 2.1 must become “obs_value”). 1396 1396 1397 -As said, the resulting SDMX definitions must be compliant with the SDMX consistency rules. For example, the SDMX DSD must have the assignmentStatus, which does not exist in VTL, the AttributeRelationship for the DataAttributes and so on. 1573 +As said, the resulting SDMX definitions must be compliant with the SDMX consistency rules. For example, the SDMX DSD must have the assignmentStatus, which does not exist in VTL, the AttributeRelationship for the DataAttributes and so on. 1398 1398 1399 - ====10.3.4.2 Unpivot Mapping====1575 +**10.3.4.2 Unpivot Mapping ** 1400 1400 1401 -An alternative mapping method from VTL to SDMX is the **Unpivot **mapping. 1577 +An alternative mapping method from VTL to SDMX is the **Unpivot **mapping. 1402 1402 1403 -Although this mapping method can be used in any case, it makes major sense in case the VTL data structure has more than one measure component (multi-measures VTL structure). For such VTL structures, in fact, the basic method cannot be applied, given that by maintaining the data structure unchanged the resulting SDMX data structure would have more than one measure component, which is not allowed by SDMX 2.1 (it allows just one measure component, the PrimaryMeasure, called “obs_value”).1579 +Although this mapping method can be used in any case, it makes major sense in case the VTL data structure has more than one measure component (multi-measures VTL structure). For such VTL structures, in fact, the basic method cannot be applied, given that by maintaining the data structure unchanged the resulting SDMX data structure would have more than one measure component, which is not allowed by SDMX 2.1 (it allows just one measure component, the PrimaryMeasure, called 1404 1404 1581 +“obs_value”). 1582 + 1405 1405 The multi-measures VTL structures have not a Measure Identifier (because the Measures are separate components) and need to be converted to SDMX dataflows having an added MeasureDimension which disambiguates the multiple measures, and an added PrimaryMeasure, in which the measures’ values are maintained. 1406 1406 1407 1407 The **unpivot** mapping behaves like follows: ... ... @@ -1408,34 +1408,43 @@ 1408 1408 1409 1409 * like in the basic mapping, a VTL (simple) identifier becomes a SDMX 1410 1410 1411 -Dimension and a VTL (time) identifier becomes a SDMX TimeDimension (as said, a measure identifier cannot exist in multi-measure VTL structures); 1589 +Dimension and a VTL (time) identifier becomes a SDMX TimeDimension (as said, a measure identifier cannot exist in multi-measure VTL structures); 1412 1412 1413 1413 * a MeasureDimension component called “measure_name” is added to the SDMX DataStructure; 1414 -* a PrimaryMeasure component called “obs_value” is added to the SDMX DataStructure; 1415 -* each VTL Measure is mapped to a Concept of the SDMX MeasureDimension having the same name as the VTL Measure (therefore all the VTL Measure Components do not originate Components in the SDMX DataStructure); 1416 -* a VTL Attribute becomes a SDMX DataAttribute having AttributeRelationship referred to all the SDMX DimensionComponents including the TimeDimension and except the MeasureDimension. 1592 +* a PrimaryMeasure component called “obs_value” is added to the SDMX DataStructure; 1593 +* each VTL Measure is mapped to a Concept of the SDMX MeasureDimension having the same name as the VTL Measure (therefore all the VTL Measure Components do not originate Components in the SDMX DataStructure); 1594 +* a VTL Attribute becomes a SDMX DataAttribute having AttributeRelationship referred to all the SDMX DimensionComponents including the TimeDimension and except the MeasureDimension. 1417 1417 1418 1418 The summary mapping table of the **unpivot** mapping method is the following: 1419 1419 1420 -(% style="width:904.294px" %) 1421 -|(% style="width:291px" %)**VTL**|(% style="width:611px" %)**SDMX** 1422 -|(% style="width:291px" %)(Simple) Identifier|(% style="width:611px" %)Dimension 1423 -|(% style="width:291px" %)(Time) Identifier|(% style="width:611px" %)TimeDimension 1424 -|(% style="width:291px" %)All Measure Components|(% style="width:611px" %)((( 1425 -MeasureDimension (having one Measure Concept for each VTL measure component) & PrimaryMeasure 1598 + 1599 +|VTL|SDMX 1600 +|(Simple) Identifier|Dimension 1601 +|(Time) Identifier|TimeDimension 1602 +|All Measure Components|((( 1603 +MeasureDimension (having one Measure Concept for each VTL measure component) & 1604 + 1605 +PrimaryMeasure 1426 1426 ))) 1427 -|(% style="width:291px" %)Attribute |(% style="width:611px" %)((( 1428 -DataAttribute depending on all SDMX Dimensions including the TimeDimension and except the MeasureDimension 1607 +|Attribute |((( 1608 +DataAttribute depending on all 1609 + 1610 +SDMX Dimensions including the 1611 + 1612 +TimeDimension and except the MeasureDimension 1429 1429 ))) 1430 1430 1431 1431 At observation / data point level: 1432 1432 1433 -* a multi-measure VTL Data Point becomes a set of SDMX observations, one for each VTL measure 1434 -* the values of the VTL identifiers become the values of the corresponding SDMX Dimensions, for all the observations of the set above 1435 -* the name of the j^^th^^ VTL measure (e.g. “Cj”) becomes the value of the SDMX MeasureDimension of the j^^th^^ observation of the set (i.e. the Concept Cj) 1436 -* the value of the j^^th^^ VTL measure becomes the value of the SDMX PrimaryMeasure of the j^^th^^ observation of the set 1437 -* the values of the VTL Attributes become the values of the corresponding SDMX DataAttributes (in principle for all the observations of the set above) 1617 + a multi-measure VTL Data Point becomes a set of SDMX observations, one for each VTL measure 1438 1438 1619 + the values of the VTL identifiers become the values of the corresponding SDMX Dimensions, for all the observations of the set above 1620 + 1621 +* 1622 +** the name of the j^^th^^ VTL measure (e.g. “Cj”) becomes the value of the SDMX MeasureDimension of the j^^th^^ observation of the set (i.e. the Concept Cj) 1623 +** the value of the j^^th^^ VTL measure becomes the value of the SDMX PrimaryMeasure of the j^^th^^ observation of the set 1624 +** the values of the VTL Attributes become the values of the corresponding SDMX DataAttributes (in principle for all the observations of the set above) 1625 + 1439 1439 If desired, this method can be applied also to mono-measure VTL structures, provided that none of the VTL components has already the role of measure identifier. 1440 1440 1441 1441 Like in the general case, a MeasureDimension component called “measure_name” would be added to the SDMX DataStructure and would have just one possible measure concept, corresponding to the unique VTL measure. The original VTL measure component would not become a Component in the SDMX data structure. The value of the VTL measure would be assigned to the SDMX PrimaryMeasure called “obs_value”. ... ... @@ -1442,150 +1442,219 @@ 1442 1442 1443 1443 In any case, the resulting SDMX definitions must be compliant with the SDMX consistency rules. For example, the possible Concepts of the SDMX MeasureDimension need to be listed in a SDMX ConceptScheme, with proper id, agency and version; moreover, the SDMX DSD must have the assignmentStatus, which does not exist in VTL, the attributeRelationship for the DataAttributes and so on. 1444 1444 1445 - ====10.3.4.3 From VTL Measures to SDMX Data Attributes****====1632 +**10.3.4.3 From VTL Measures to SDMX Data Attributes ** 1446 1446 1447 1447 For the multi-measure VTL structures (having more than one Measure Component), it may happen that the Measures of the VTL Data Structure need to be managed as DataAttributes in SDMX. Therefore a third mapping method consists in transforming one VTL measure in the SDMX primaryMeasure and all the other VTL Measures in SDMX DataAttributes. This method is called M2A (“M2A” stands for “Measures to DataAttributes”). 1448 1448 1449 1449 When applied to mono-measure VTL structures (having one Measure component), the M2A method behaves like the Basic mapping (the VTL Measure component becomes the SDMX primary measure “obs_value”, there is no additional VTL measure to be converted to SDMX DataAttribute). Therefore the mapping table is the same as for the Basic method: 1450 1450 1451 -(% style="width:591.294px" %) 1452 -|(% style="width:252px" %)**VTL**|(% style="width:336px" %)**SDMX** 1453 -|(% style="width:252px" %)(Simple) Identifier|(% style="width:336px" %)Dimension 1454 -|(% style="width:252px" %)(Time) Identifier|(% style="width:336px" %)TimeDimension 1455 -|(% style="width:252px" %)(Measure) Identifier (if any)|(% style="width:336px" %)MeasureDimension 1456 -|(% style="width:252px" %)Measure|(% style="width:336px" %)PrimaryMeasure 1457 -|(% style="width:252px" %)Attribute|(% style="width:336px" %)DataAttribute 1638 +|VTL|SDMX 1639 +|(Simple) Identifier|Dimension 1640 +|(Time) Identifier|TimeDimension 1641 +|(Measure) Identifier (if any)|MeasureDimension 1642 +|Measure|PrimaryMeasure 1643 +|Attribute|DataAttribute 1458 1458 1459 -For multi-measure VTL structures (having more than one Measure component), one VTL Measure becomes the SDMX PrimaryMeasure while the other VTL Measures maintain their names and values but assume the role of DataAttribute in SDMX. The choice of the VTL Measure that correspond to the SDMX PrimaryMeasure is left to the definer of the SDMX data structure definition. 1645 +For multi-measure VTL structures (having more than one Measure component), one VTL Measure becomes the SDMX PrimaryMeasure while the other VTL Measures maintain their names and values but assume the role of DataAttribute in SDMX. The choice of the VTL Measure that correspond to the SDMX PrimaryMeasure is left to the definer of the SDMX data structure definition. 1460 1460 1461 -Taking into account that the multi-measure VTL structures do not have a measure identifier, the mapping table is the following: 1647 +2Taking into account that the multi-measure VTL structures do not have a measure 2073 identifier, the mapping table is the following: 1462 1462 1463 -(% style="width:588.294px" %) 1464 -|(% style="width:259px" %)**VTL**|(% style="width:326px" %)**SDMX** 1465 -|(% style="width:259px" %)(Simple) Identifier|(% style="width:326px" %)Dimension 1466 -|(% style="width:259px" %)(Time) Identifier|(% style="width:326px" %)TimeDimension 1467 -|(% style="width:259px" %)One of the Measures|(% style="width:326px" %)PrimaryMeasure 1468 -|(% style="width:259px" %)Other Measures|(% style="width:326px" %)DataAttribute 1469 -|(% style="width:259px" %)Attribute|(% style="width:326px" %)DataAttribute 1649 +|VTL|SDMX 1650 +|(Simple) Identifier|Dimension 1651 +|(Time) Identifier|TimeDimension 1652 +|One of the Measures|PrimaryMeasure 1653 +|Other Measures|DataAttribute 1654 +|Attribute|DataAttribute 1470 1470 1471 -Even in this case, the resulting SDMX definitions must be compliant with the SDMX consistency rules. For example, the SDMX DSD must have the assignmentStatus, which does not exist in VTL, the attributeRelationship for the DataAttributes and so on. In particular, the primaryMeasure of the SDMX 2.1 DSD must be called “obs_value” and must be one of the VTL Measures, chosen by the DSD definer. 1656 +Even in this case, the resulting SDMX definitions must be compliant with the SDMX consistency rules. For example, the SDMX DSD must have the assignmentStatus, which does not exist in VTL, the attributeRelationship for the DataAttributes and so on. In particular, the primaryMeasure of the SDMX 2.1 DSD must be called “obs_value” and must be one of the VTL Measures, chosen by the DSD definer. 1472 1472 1473 1473 === 10.3.5 Declaration of the mapping methods between data structures === 1474 1474 1475 1475 In order to define and understand properly VTL transformations, the applied mapping methods must be specified in the SDMX structural metadata. If the default mapping method (Basic) is applied, no specification is needed. 1476 1476 1662 + 1477 1477 A customized mapping can be defined through the VtlMappingScheme and VtlDataflowMapping classes (see the section of the SDMX IM relevant to the VTL). A VtlDataflowMapping allows specifying the mapping methods to be used for a specific dataflow, both in the direction from SDMX to VTL (toVtlMappingMethod) and from VTL to SDMX (fromVtlMappingMethod); in fact a VtlDataflowMapping associates the structured URN that identifies a SDMX dataflow to its VTL alias and its mapping methods. 1478 1478 1479 -It is possible to specify the toVtlMappingMethod and fromVtlMappingMethod also for the conventional dataflow called “generic_dataflow”: in this case the specified mapping methods are intended to become the default ones, overriding the “Basic” methods. In turn, the toVtlMappingMethod and fromVtlMappingMethod declared for a specific Dataflow are intended to override the default ones for such a Dataflow.1665 +It is possible to specify the toVtlMappingMethod and fromVtlMappingMethod also for the conventional dataflow called “generic_dataflow”: in this case the specified mapping methods are intended to become the default ones, overriding the 1480 1480 1667 +“Basic” methods. In turn, the toVtlMappingMethod and fromVtlMappingMethod declared for a specific Dataflow are intended to override the default ones for such a Dataflow. 1668 + 1481 1481 The VtlMappingScheme is a container for zero or more VtlDataflowMapping (besides possible mappings to artefacts other than dataflows). 1482 1482 1483 -=== 10.3.6 Mapping dataflow subsets to distinct VTL data sets[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^**~[25~]**^^>>path:#_ftn25]](%%) ===1671 +=== 10.3.6 Mapping dataflow subsets to distinct VTL data sets[[(% class="wikiinternallink wikiinternallink" %)^^**~[25~]**^^>>path:#_ftn25]](%%) === 1484 1484 1485 -Until now it as been assumed to map one SMDX Dataflow to one VTL dataset and vice-versa. This mapping one-to-one is not mandatory according to VTL because a VTL data set is meant to be a set of observations (data points) on a logical plane, having the same logical data structure and the same general meaning, independently of the possible physical representation or storage (see VTL 2.0 User Manual page 24), therefore a SDMX Dataflow can be seen either as a unique set of data observations (corresponding to one VTL data set) or as the union of many sets of data observations (each one corresponding to a distinct VTL data set).1673 +Until now it as been assumed to map one SMDX Dataflow to one VTL dataset and vice-versa. This mapping one-to-one is not mandatory according to VTL because a VTL data set is meant to be a set of observations (data points) on a logical plane, having the same logical data structure and the same general meaning, independently of the possible physical representation or storage (see VTL 2.0 User Manual page 1486 1486 1487 - Asa matteroffact, in somecasesit can beuseful to defineVTL operationsinvolvingdefiniteparts of aSDMX Dataflowinsteadthanthe whole.[[(%class="wikiinternallink wikiinternallink wikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallink wikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallinkwikiinternallink"%)^^~[26~]^^>>path:#_ftn26]](%%)1675 +24), therefore a SDMX Dataflow can be seen either as a unique set of data observations (corresponding to one VTL data set) or as the union of many sets of data observations (each one corresponding to a distinct VTL data set). 1488 1488 1489 - Therefore, in ordertomakethecodingofVTL operationssimpler whenapplied onparts ofSDMX Dataflows,itis allowedto mapdistinct parts of a SDMX Dataflowto distinct VTL datasetsaccordingto the following rulesandconventions. This kind of mapping is possiblebothfrom SDMX to VTL and from VTL to SDMX, as better explained below.[[(% class="wikiinternallink wikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[27~]^^>>path:#_ftn27]](%%)1677 +As a matter of fact, in some cases it can be useful to define VTL operations involving definite parts of a SDMX Dataflow instead than the whole.[[(% class="wikiinternallink wikiinternallink" %)^^~[26~]^^>>path:#_ftn26]](%%) 1490 1490 1491 - Given a SDMX DataflowandsomepredefinedDimensions ofitsDataStructure, it is allowed to map thesubsets of observationsthathavethe samecombination ofvaluesforsuchDimensionstocorrespondentVTLdatasets.1679 +Therefore, in order to make the coding of VTL operations simpler when applied on parts of SDMX Dataflows, it is allowed to map distinct parts of a SDMX Dataflow to distinct VTL data sets according to the following rules and conventions. This kind of mapping is possible both from SDMX to VTL and from VTL to SDMX, as better explained below.[[(% class="wikiinternallink wikiinternallink" %)^^~[27~]^^>>path:#_ftn27]](%%) 1492 1492 1493 - Forexample,assumingthattheSDMXdataflowDF1(1.0)hastheDimensionsINDICATOR,TIME_PERIODandCOUNTRY,andthatthe user declares the DimensionsINDICATORandCOUNTRYasbasisforthemapping (i.e.the mapping dimensions): theobservationsthathavethesamevalues forINDICATORandCOUNTRYwould be mappedto thesame VTL dataset (and vice-versa).1681 + Given a SDMX Dataflow and some predefined Dimensions of its 1494 1494 1683 +DataStructure, it is allowed to map the subsets of observations that have the same combination of values for such Dimensions to correspondent VTL datasets. 1684 + 1685 +For example, assuming that the SDMX dataflow DF1(1.0) has the Dimensions INDICATOR, TIME_PERIOD and COUNTRY, and that the user declares the 1686 + 1687 +Dimensions INDICATOR and COUNTRY as basis for the mapping (i.e. the mapping dimensions): the observations that have the same values for INDICATOR and COUNTRY would be mapped to the same VTL dataset (and vice-versa). 1688 + 1495 1495 In practice, this kind mapping is obtained like follows: 1496 1496 1497 -* For a given SDMX dataflow, the user (VTL definer) declares the dimension components on which the mapping will be based, in a given order.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[28~]^^>>path:#_ftn28]](%%) Following the example above, imagine that the user declares the dimensions INDICATOR and COUNTRY.1691 +* For a given SDMX dataflow, the user (VTL definer) declares the dimension components on which the mapping will be based, in a given order.[[(% class="wikiinternallink wikiinternallink" %)^^~[28~]^^>>path:#_ftn28]](%%) Following the example above, imagine that the user declares the dimensions INDICATOR and COUNTRY. 1498 1498 * The VTL dataset is given a name using a special notation also called “ordered concatenation” and composed of the following parts: 1499 -** The reference to the SDMX dataflow (expressed according to the rules described in the previous paragraphs, i.e. URN, abbreviated URN or another alias); for example DF(1.0); 1500 -** a slash (“/”) as a separator; [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[29~]^^>>path:#_ftn29]] 1501 -** The reference to a specific part of the SDMX dataflow above, expressed as the concatenation of the values that the SDMX dimensions declared above must have, separated by dots (“.”) and written in the order in which these dimensions are defined[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[30~]^^>>path:#_ftn30]](%%). For example POPULATION.USA would mean that such a VTL dataset is mapped to the SDMX observations for which the dimension //INDICATOR// is equal to POPULATION and the dimension //COUNTRY// is equal to USA. 1693 +** The reference to the SDMX dataflow (expressed according to the rules described in the previous paragraphs, i.e. URN, abbreviated 1502 1502 1695 +URN or another alias); for example DF(1.0); o a slash (“/”) as a separator; [[(% class="wikiinternallink wikiinternallink" %)^^~[29~]^^>>path:#_ftn29]] 1696 + 1697 +* 1698 +** The reference to a specific part of the SDMX dataflow above, expressed as the concatenation of the values that the SDMX dimensions declared above must have, separated by dots (“.”) and written in the order in which these dimensions are defined[[(% class="wikiinternallink wikiinternallink" %)^^~[30~]^^>>path:#_ftn30]](%%) . For example POPULATION.USA would mean that such a VTL dataset is mapped to the SDMX observations for which the dimension //INDICATOR// is equal to POPULATION and the dimension //COUNTRY// is equal to USA. 1699 + 1503 1503 In the VTL transformations, this kind of dataset name must be referenced between single quotes because the slash (“/”) is not a regular character according to the VTL rules. 1504 1504 1505 1505 Therefore, the generic name of this kind of VTL datasets would be: 1506 1506 1507 - >‘DF(1.0)///INDICATORvalue//.//COUNTRYvalue//’1704 +‘DF(1.0)///INDICATORvalue//.//COUNTRYvalue//’ 1508 1508 1509 1509 Where DF(1.0) is the Dataflow and //INDICATORvalue// and //COUNTRYvalue //are placeholders for one value of the INDICATOR and // //COUNTRY dimensions. 1510 1510 1511 1511 Instead the specific name of one of these VTL datasets would be: 1512 1512 1513 - >‘DF(1.0)/POPULATION.USA’1710 +‘DF(1.0)/POPULATION.USA’ 1514 1514 1515 -In particular, this is the VTL dataset that contains all the observations of the dataflow DF(1.0) for which //INDICATOR// = POPULATION and //COUNTRY// = USA. 1712 +In particular, this is the VTL dataset that contains all the observations of the dataflow DF(1.0) for which //INDICATOR// = POPULATION and //COUNTRY// = USA. 1516 1516 1517 1517 Let us now analyse the different meaning of this kind of mapping in the two mapping directions, i.e. from SDMX to VTL and from VTL to SDMX. 1518 1518 1519 -As already said, the mapping from SDMX to VTL happens when the VTL datasets are operand of VTL transformations, instead the mapping from VTL to SDMX happens when the VTL datasets are result of VTL transformations[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[31~]^^>>path:#_ftn31]](%%) and need to be treated as SDMX objects. This kind of mapping can be applied independently in the two directions and the Dimensions on which the mapping is based can be different in the two directions: these Dimensions are defined in the ToVtlSpaceKey and in the FromVtlSpaceKey classes respectively.1716 +As already said, the mapping from SDMX to VTL happens when the VTL datasets are operand of VTL transformations, instead the mapping from VTL to SDMX happens when the VTL datasets are result of VTL transformations[[(% class="wikiinternallink wikiinternallink" %)^^~[31~]^^>>path:#_ftn31]](%%) and need to be treated as SDMX objects. This kind of mapping can be applied independently in the two directions and the Dimensions on which the mapping is based can be different in the two directions: these Dimensions are defined in the ToVtlSpaceKey and in the FromVtlSpaceKey classes respectively. 1520 1520 1521 -First, let us see what happens in the __mapping direction from SDMX to VTL__, i.e. when parts of a SDMX dataflow (e.g. DF1(1.0)) need to be mapped to distinct VTL datasets that are operand of some VTL transformations.1718 +First, let us see what happens in the mapping direction from SDMX to VTL, i.e. when parts of a SDMX dataflow (e.g. DF1(1.0)) need to be mapped to distinct VTL datasets that are operand of some VTL transformations. 1522 1522 1523 -As already said, each VTL dataset is assumed to contain all the observations of the SDMX dataflow having INDICATOR=//INDICATORvalue //and COUNTRY=//COUNTRYvalue//. For example, the VTL dataset ‘DF1(1.0)/POPULATION.USA’ would contain all the observations of DF1(1.0) having INDICATOR = POPULATION and COUNTRY = USA.1720 +As already said, each VTL dataset is assumed to contain all the observations of the 1524 1524 1525 - In order to obtain the data structure of these VTL datasets from theSDMXone, it is assumedthatthe SDMX dimensions onwhichthe mappingis based are dropped, i.e. not maintained in the VTL data structure; this is possible because theirvaluesare fixed for each one of the invokedVTL datasets[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[32~]^^>>path:#_ftn32]](%%). After that, the mapping method from SDMX to VTL specified for the dataflow DF1(1.0) is applied (i.e. basic, pivot …).1722 +SDMX dataflow having INDICATOR=//INDICATORvalue //and COUNTRY= 1526 1526 1527 - In the exampleabove,for allthe datasetsof the kind‘DF1(1.0)///INDICATORvalue//.//COUNTRYvalue//’,the dimensions INDICATOR and COUNTRYwouldbe dropped sothatthe data structure of all theresulting VTL datasetswouldhave theidentifierTIME_PERIODonly.1724 +//COUNTRYvalue//. For example, the VTL dataset ‘DF1(1.0)/POPULATION.USA’ would contain all the observations of DF1(1.0) having INDICATOR = POPULATION and COUNTRY = USA. 1528 1528 1726 +In order to obtain the data structure of these VTL datasets from the SDMX one, it is assumed that the SDMX dimensions on which the mapping is based are dropped, i.e. not maintained in the VTL data structure; this is possible because their values are fixed for each one of the invoked VTL datasets[[(% class="wikiinternallink wikiinternallink" %)^^~[32~]^^>>path:#_ftn32]](%%). After that, the mapping method from SDMX to VTL specified for the dataflow DF1(1.0) is applied (i.e. basic, pivot …). 1727 + 1728 +In the example above, for all the datasets of the kind 1729 + 1730 +‘DF1(1.0)///INDICATORvalue//.//COUNTRYvalue//’, the dimensions INDICATOR and COUNTRY would be dropped so that the data structure of all the resulting VTL data sets would have the identifier TIME_PERIOD only. 1731 + 1529 1529 It should be noted that the desired VTL datasets (i.e. of the kind ‘DF1(1.0)/// INDICATORvalue//.//COUNTRYvalue//’) can be obtained also by applying the VTL operator “**sub**” (subspace) to the dataflow DF1(1.0), like in the following VTL expression: 1530 1530 1531 -> ‘DF1(1.0)/POPULATION.USA’ := 1532 -> DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA” ]; 1533 -> ‘DF1(1.0)/POPULATION.CANADA’ := 1534 -> DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“CANADA” ]; 1535 -> … … … 1734 +‘DF1(1.0)/POPULATION.USA’ := 1536 1536 1537 - Infactthe VTL operator “sub”has exactly the same behaviour. Therefore, mapping different parts of a SDMX dataflow to different VTLdatasets in the direction from SDMX to VTLthrough the ordered concatenation notation is equivalent to a proper use of the operator“**sub**”on such a dataflow. [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[33~]^^>>path:#_ftn33]]1736 +DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA” ]; 1538 1538 1738 + 1739 +‘DF1(1.0)/POPULATION.CANADA’ := 1740 + 1741 +DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“CANADA” ]; 1742 + 1743 + 1744 +… … … 1745 + 1746 +In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX dataflow to different VTL datasets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a dataflow. [[(% class="wikiinternallink wikiinternallink" %)^^~[33~]^^>>path:#_ftn33]] 1747 + 1539 1539 In the direction from SDMX to VTL it is allowed to omit the value of one or more Dimensions on which the mapping is based, but maintaining all the separating dots (therefore it may happen to find two or more consecutive dots and dots in the beginning or in the end). The absence of value means that for the corresponding Dimension all the values are kept and the Dimension is not dropped. 1540 1540 1541 -For example, ‘DF(1.0)/POPULATION.’ (note the dot in the end of the name) is the VTL dataset that contains all the observations of the dataflow DF(1.0) for which //INDICATOR// = POPULATION and COUNTRY = any value. 1750 +For example, ‘DF(1.0)/POPULATION.’ (note the dot in the end of the name) is the VTL dataset that contains all the observations of the dataflow DF(1.0) for which //INDICATOR// = POPULATION and COUNTRY = any value. 1542 1542 1543 1543 This is equivalent to the application of the VTL “sub” operator only to the identifier //INDICATOR//: 1544 1544 1545 -> ‘DF1(1.0)/POPULATION.’ := 1546 -> DF1(1.0) [sub INDICATOR=“POPULATION” ]; 1754 +‘DF1(1.0)/POPULATION.’ := 1547 1547 1756 +DF1(1.0) [ sub INDICATOR=“POPULATION” ]; 1757 + 1758 + 1548 1548 Therefore the VTL dataset ‘DF1(1.0)/POPULATION.’ would have the identifiers COUNTRY and TIME_PERIOD. 1549 1549 1550 1550 Heterogeneous invocations of the same Dataflow are allowed, i.e. omitting different Dimensions in different invocations. 1551 1551 1552 -Let us now analyse the __mapping direction from VTL to SDMX__.1763 +Let us now analyse the mapping direction from VTL to SDMX. 1553 1553 1554 1554 In this situation, distinct parts of a SDMX dataflow are calculated as distinct VTL datasets, under the constraint that they must have the same VTL data structure. 1555 1555 1556 1556 For example, let us assume that the VTL programmer wants to calculate the SDMX dataflow DF2(1.0) having the Dimensions TIME_PERIOD, INDICATOR, and COUNTRY and that such a programmer finds it convenient to calculate separately the parts of DF2(1.0) that have different combinations of values for INDICATOR and COUNTRY: 1557 1557 1558 -* each part is calculated as a VTL derived dataset, result of a dedicated VTL transformation; [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[34~]^^>>path:#_ftn34]](%%)1559 -* the data structure of all these VTL datasets has the TIME_PERIOD identifier and does not have the INDICATOR and COUNTRY identifiers.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[35~]^^>>path:#_ftn35]]1769 +* each part is calculated as a VTL derived dataset, result of a dedicated VTL transformation; [[(% class="wikiinternallink wikiinternallink" %)^^~[34~]^^>>path:#_ftn34]](%%) 1770 +* the data structure of all these VTL datasets has the TIME_PERIOD identifier and does not have the INDICATOR and COUNTRY identifiers.[[(% class="wikiinternallink wikiinternallink" %)^^~[35~]^^>>path:#_ftn35]] 1560 1560 1561 -Under these hypothesis, such derived VTL datasets can be mapped to DF2(1.0) by declaring the Dimensions INDICATOR and COUNTRY as mapping dimensions[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[36~]^^>>path:#_ftn36]](%%).1772 +Under these hypothesis, such derived VTL datasets can be mapped to DF2(1.0) by declaring the Dimensions INDICATOR and COUNTRY as mapping dimensions[[(% class="wikiinternallink wikiinternallink" %)^^~[36~]^^>>path:#_ftn36]](%%). 1562 1562 1563 -The corresponding VTL transformations, assuming that the result needs to be persistent, would be of this kind:^^ ^^[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[37~]^^>>path:#_ftn37]]1774 +The corresponding VTL transformations, assuming that the result needs to be persistent, would be of this kind:^^ ^^[[(% class="wikiinternallink wikiinternallink" %)^^~[37~]^^>>path:#_ftn37]] 1564 1564 1565 1565 ‘DF2(1.0)///INDICATORvalue//.//COUNTRYvalue//’ <- expression 1566 1566 1567 1567 Some examples follow, for some specific values of INDICATOR and COUNTRY: 1568 1568 1569 -‘DF2(1.0)/GDPPERCAPITA.USA’ <- expression11; 1780 + ‘DF2(1.0)/GDPPERCAPITA.USA’ <- expression11; 1781 + 1570 1570 ‘DF2(1.0)/GDPPERCAPITA.CANADA’ <- expression12; 1783 + 1571 1571 … … … 1572 -‘DF2(1.0)/POPGROWTH.USA’ <- expression21; 1573 -‘DF2(1.0)/POPGROWTH.CANADA’ <- expression22; 1574 1574 1786 + ‘DF2(1.0)/POPGROWTH.USA’ <- expression21; 1787 + 1788 + ‘DF2(1.0)/POPGROWTH.CANADA’ <- expression22; 1789 + 1575 1575 … … … 1576 1576 1577 -As said, it is assumed that these VTL derived datasets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 1578 1578 1579 - [[image:1747859458410-183.png||height="170"width="663"]]1793 +As said, it is assumed that these VTL derived datasets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 1580 1580 1581 -It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0)), that can be mapped one-to-one to the homonymous SDMX Dataflow. Following the same example, these VTL transformations would be: 1795 +|((( 1796 + //VTL dataset // 1582 1582 1583 -[[image:1747859612718-454.png||height="451" width="602"]] 1798 + 1799 +)))|(% colspan="2" %)//INDICATOR value //|(% colspan="2" %)//COUNTRY value// 1800 +|‘DF2(1.0)/GDPPERCAPITA.USA’ |GDPPERCAPITA| | |USA 1801 +|((( 1802 +‘DF2(1.0)/GDPPERCAPITA.CANADA’ 1584 1584 1585 -In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent data sets are united and give the final result DF2(1.0)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[38~]^^>>path:#_ftn38]](%%), which can be mapped one-to-one to the homonymous SDMX dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. 1804 +… … … 1805 +)))|GDPPERCAPITA| | |CANADA 1806 +|‘DF2(1.0)/POPGROWTH.USA’ |POPGROWTH | | |USA 1807 +|((( 1808 +‘DF2(1.0)/POPGROWTH.CANADA’ 1586 1586 1587 -Therefore, mapping different VTL datasets having the same data structure to different parts of a SDMX dataflow, i.e. in the direction from VTL to SDMX, through the ordered concatenation notation is equivalent to a proper use of the operators “calc” and “union” on such datasets. [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[39~]^^>>path:#_ftn39]](%%)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[40~]^^>>path:#_ftn40]] 1810 +… … … 1811 +)))|POPGROWTH | | |CANADA 1588 1588 1813 +It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0)), that can be mapped one-to-one to the homonymous SDMX Dataflow. Following the same example, these VTL transformations would be: 1814 + 1815 +DF2bis_GDPPERCAPITA_USA := ‘DF2(1.0)/GDPPERCAPITA.USA’ 1816 + 1817 +[calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”]; 1818 + 1819 +DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”]; … … … 1820 + 1821 +DF2bis_POPGROWTH_USA := ‘DF2(1.0)/POPGROWTH.USA’ 1822 + 1823 +[calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY :=”USA”]; 1824 + 1825 +DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0)/POPGROWTH.CANADA’ 1826 + 1827 +[calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”]; … … … 1828 + 1829 +DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, 1830 + 1831 +DF2bis_GDPPERCAPITA_CANADA’, 1832 + 1833 +… , 1834 + 1835 +DF2bis_POPGROWTH_USA’, 1836 + 1837 +DF2bis_POPGROWTH_CANADA’ 1838 + 1839 +…); 1840 + 1841 +In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent data sets are united and give the final result DF2(1.0)[[(% class="wikiinternallink wikiinternallink" %)^^~[38~]^^>>path:#_ftn38]](%%), which can be mapped one-to-one to the homonymous SDMX dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. 1842 + 1843 +Therefore, mapping different VTL datasets having the same data structure to different parts of a SDMX dataflow, i.e. in the direction from VTL to SDMX, through the ordered concatenation notation is equivalent to a proper use of the operators “calc” and “union” on such datasets. [[(% class="wikiinternallink wikiinternallink" %)^^~[39~]^^>>path:#_ftn39]](%%)[[(% class="wikiinternallink wikiinternallink" %)^^~[40~]^^>>path:#_ftn40]] 1844 + 1589 1589 It is worth noting that in the direction from VTL to SDMX it is mandatory to specify the value for every Dimension on which the mapping is based (in other word, in the name of the calculated VTL dataset is not possible to omit the value of some of the Dimensions). 1590 1590 1591 1591 === 10.3.7 Mapping variables and value domains between VTL and SDMX === ... ... @@ -1592,41 +1592,58 @@ 1592 1592 1593 1593 With reference to the VTL “model for Variables and Value domains”, the following additional mappings have to be considered: 1594 1594 1595 -(% style="width:890.835px" %) 1596 -|(% style="width:314px" %)VTL|(% style="width:574px" %)SDMX 1597 -|(% style="width:314px" %)**Data Set Component**|(% style="width:574px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a Dimension or a PrimaryMeasure or a DataAttribute) belonging to one specific Dataflow^^42^^ 1598 -|(% style="width:314px" %)**Represented Variable**|(% style="width:574px" %)**Concept** with a definite Representation 1599 -|(% style="width:314px" %)**Value Domain**|(% style="width:574px" %)**Representation** (see the Structure Pattern in the Base Package) 1600 -|(% style="width:314px" %)**Enumerated Value Domain / Code List**|(% style="width:574px" %)((( 1601 -**Codelist** (for enumerated Dimension, PrimaryMeasure, DataAttribute) or **ConceptScheme **(for MeasureDimension) 1851 +|VTL|SDMX 1852 +|**Data Set Component**|Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a Dimension or a PrimaryMeasure or a DataAttribute) belonging to one specific Dataflow^^42^^ 1853 +|**Represented Variable**|**Concept** with a definite Representation 1854 +|**Value Domain**|**Representation** (see the Structure Pattern in the Base Package) 1855 +|**Enumerated Value Domain / Code List**|((( 1856 +**Codelist** (for enumerated 1857 + 1858 +Dimension, PrimaryMeasure, 1859 + 1860 +DataAttribute) or **ConceptScheme** 1861 + 1862 +(for MeasureDimension) 1602 1602 ))) 1603 -|(% style="width:314px" %)**Code**|(% style="width:574px" %)**Code** (for enumerated Dimension, PrimaryMeasure, DataAttribute) or **Concept** (for MeasureDimension) 1604 -|(% style="width:314px" %)**Described Value Domain**|(% style="width:574px" %)((( 1605 -non-enumerated** Representation **(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 1864 +|**Code**|**Code** (for enumerated Dimension, PrimaryMeasure, DataAttribute) or **Concept** (for MeasureDimension) 1865 +|**Described Value Domain**|((( 1866 +non-enumerated** Representation** 1867 + 1868 +(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 1606 1606 ))) 1607 -|(% style="width:314px" %)**Value**|(% style="width:574px" %)((( 1608 -Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or to a valid **value **(for non-enumerated** **Representations) or to a **Concept **(for MeasureDimension) 1870 +|**Value**|((( 1871 +Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a 1872 + 1873 +Codelist (for enumerated 1874 + 1875 +Representations) or to a valid **value **(for non-enumerated** ** 1876 + 1877 +Representations) or to a **Concept** 1878 + 1879 +(for MeasureDimension) 1609 1609 ))) 1610 -| (% style="width:314px" %)**Value Domain Subset / Set**|(% style="width:574px" %)This abstraction does not exist in SDMX1611 -| (% style="width:314px" %)**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:574px" %)This abstraction does not exist in SDMX1612 -| (% style="width:314px" %)**Described Value Domain Subset / Described Set**|(% style="width:574px" %)This abstraction does not exist in SDMX1613 -| (% style="width:314px" %)**Set list**|(% style="width:574px" %)This abstraction does not exist in SDMX1881 +|**Value Domain Subset / Set**|This abstraction does not exist in SDMX 1882 +|**Enumerated Value Domain Subset / Enumerated Set**|This abstraction does not exist in SDMX 1883 +|**Described Value Domain Subset / Described Set**|This abstraction does not exist in SDMX 1884 +|**Set list**|This abstraction does not exist in SDMX 1614 1614 1615 1615 The main difference between VTL and SDMX relies on the fact that the VTL artefacts for defining subsets of Value Domains do not exist in SDMX, therefore the VTL features for referring to predefined subsets are not available in SDMX. These artefacts are the Value Domain Subset (or Set), either enumerated or described, the Set List (list of values belonging to enumerated subsets) and the Data Set Component (aimed at defining the set of values that the Component of a Data Set can take, possibly a subset of the codes of Value Domain). 1616 1616 1617 -Another difference consists in the fact that all Value Domains are considered as identifiable objects in VTL either if enumerated or not, while in SDMX the Codelist (corresponding to a VTL enumerated Value Domain) is identifiable, while the SDMX non-enumerated Representation (corresponding to a VTL non-enumerated Value Domain) is not identifiable. As a consequence, the definition of the VTL rulesets, which in VTL can refer either to enumerated or non-enumerated value domains, in SDMX can refer only to enumerated Value Domains (i.e. to SDMX Codelists).1888 +Another difference consists in the fact that all Value Domains are considered as identifiable objects in VTL either if enumerated or not, while in SDMX the Codelist (corresponding to a VTL enumerated Value Domain) is identifiable, while the SDMX non-enumerated Representation (corresponding to a VTL non-enumerated Value 1618 1618 1619 - As for themappingbetween VTL variablesand SDMX Concepts,it shouldbenoted that theseartefactsdo notcoincideperfectly. Infact, theVTL variables are representedvariables, defined always on the same Value Domain (“Representation”in SDMX) independentlyof thedataset/ datastructureinwhichthey appear[[(% class="wikiinternallinkwikiinternallinkwikiinternallinkwikiinternallink wikiinternallink wikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallink wikiinternallink wikiinternallinkwikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallink"%)^^~[41~]^^>>path:#_ftn41]](%%), while theSDMXConcepts canhavedifferentRepresentations in different DataStructures.[[(% class="wikiinternallinkwikiinternallinkwikiinternallink wikiinternallink wikiinternallinkwikiinternallink wikiinternallinkwikiinternallinkwikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[42~]^^>>path:#_ftn42]](%%) This meansthatoneSDMX Concept can correspondto many VTL Variables, one for each representation the Concept has.1890 +Domain) is not identifiable. As a consequence, the definition of the VTL rulesets, which in VTL can refer either to enumerated or non-enumerated value domains, in SDMX can refer only to enumerated Value Domains (i.e. to SDMX Codelists). 1620 1620 1621 - Therefore,it isimportant to beaware that someVTL operations(forexample thebinaryoperationsat datasetlevel)areconsistentonlyifthecomponentshavingthesamenames in theoperated VTL datasetshavealsothe same representation(i.e.thesameValueDomain asforVTL). Forexample,itis possibletoobtain correct resultsfromtheVTLexpression1892 +As for the mapping between VTL variables and SDMX Concepts, it should be noted that these artefacts do not coincide perfectly. In fact, the VTL variables are represented variables, defined always on the same Value Domain (“Representation” in SDMX) independently of the data set / data structure in which they appear[[(% class="wikiinternallink wikiinternallink" %)^^~[41~]^^>>path:#_ftn41]](%%), while the SDMX Concepts can have different Representations in different DataStructures.[[(% class="wikiinternallink wikiinternallink" %)^^~[42~]^^>>path:#_ftn42]](%%) This means that one SDMX Concept can correspond to many VTL Variables, one for each representation the Concept has. 1622 1622 1623 - DS_c := DS_a+DS_b(whereDS_a,DS_b,DS_care VTLDataSets)1894 +Therefore, it is important to be aware that some VTL operations (for example the binary operations at data set level) are consistent only if the components having the same names in the operated VTL data sets have also the same representation (i.e. the same Value Domain as for VTL). For example, it is possible to obtain correct results from the VTL expression 1624 1624 1625 - ifthematchingcomponentsinDS_aandDS_b(e.g.ref_date,geo_area,sector…)refertothesamegeneralrepresentation.Insimplerwords,DS_aandDS_bmustusethesamevalues/codes(forref_date,geo_area,sector… ), otherwisetherelevantvalueswould not match and theresultof the operation would be wrong.1896 + DS_c := DS_a + DS_b (where DS_a, DS_b, DS_c are VTL Data Sets) 1626 1626 1898 +if the matching components in DS_a and DS_b (e.g. ref_date, geo_area, sector …) refer to the same general representation. In simpler words, DS_a and DS_b must use the same values/codes (for ref_date, geo_area, sector … ), otherwise the relevant values would not match and the result of the operation would be wrong. 1899 + 1627 1627 As mentioned, the property above is not enforced by construction in SDMX, and different representations of the same Concept can be not compatible one another (for example, it may happen that geo_area is represented by ISO-alpha-3 codes in DS_a and by ISO alpha-2 codes in DS_b). Therefore, it will be up to the definer of VTL transformations to ensure that the VTL expressions are consistent with the actual representations of the correspondent SDMX Concepts. 1628 1628 1629 -It remains up to the SDMX-VTL definer also the assurance of the consistency between a VTL Ruleset defined on Variables and the SDMX Components on which the Ruleset is applied. In fact, a VTL Ruleset is expressed by means of the values of the Variables (i.e. SDMX Concepts), i.e. assuming definite representations for them (e.g. ISO-alpha-3 for country). If the Ruleset is applied to SDMX Components that have the same name of the Concept they refer to but different representations (e.g. ISO-alpha-2 for country), the Ruleset cannot work properly. 1902 +It remains up to the SDMX-VTL definer also the assurance of the consistency between a VTL Ruleset defined on Variables and the SDMX Components on which the Ruleset is applied. In fact, a VTL Ruleset is expressed by means of the values of the Variables (i.e. SDMX Concepts), i.e. assuming definite representations for them (e.g. ISO-alpha-3 for country). If the Ruleset is applied to SDMX Components that have the same name of the Concept they refer to but different representations (e.g. ISO-alpha-2 for country), the Ruleset cannot work properly. 1630 1630 1631 1631 == 10.4 Mapping between SDMX and VTL Data Types == 1632 1632 ... ... @@ -1644,7 +1644,6 @@ 1644 1644 1645 1645 The VTL basic scalar types are listed below and follow a hierarchical structure in terms of supersets/subsets (e.g. “scalar” is the superset of all the basic scalar types): 1646 1646 1647 -[[image:1747859722732-549.png||height="283" width="224"]] 1648 1648 1649 1649 **Figure 13 – VTL Basic Scalar Types** 1650 1650 ... ... @@ -1670,162 +1670,208 @@ 1670 1670 1671 1671 The following table describes the default mapping for converting from the SDMX data types to the VTL basic scalar types. 1672 1672 1673 - (%style="width:653.835px" %)1674 -|( % style="width:366px" %)**SDMX data type(BasicComponentDataType)**|(% style="width:284px" %)**Default VTL basic scalar type**1675 - |(% style="width:366px"%)(((1676 - **String**1945 +|**SDMX data type (BasicComponentDataType)**|**Default VTL basic scalar type** 1946 +|((( 1947 +**String ** 1948 + 1677 1677 (string allowing any character) 1678 -)))|(% style="width:284px" %)**string** 1679 -|(% style="width:366px" %)((( 1680 -**Alpha** 1950 +)))|**string** 1951 +|((( 1952 +**Alpha ** 1953 + 1681 1681 (string which only allows A-z) 1682 -)))|(% style="width:284px" %)**string** 1683 -|(% style="width:366px" %)((( 1684 -**AlphaNumeric** 1955 +)))|**string** 1956 +|((( 1957 +**AlphaNumeric ** 1958 + 1685 1685 (string which only allows A-z and 0-9) 1686 -)))|(% style="width:284px" %)**string** 1687 -|(% style="width:366px" %)((( 1688 -**Numeric** 1960 +)))|**string** 1961 +|((( 1962 +**Numeric ** 1963 + 1689 1689 (string which only allows 0-9, but is not numeric so that is can having leading zeros) 1690 -)))|(% style="width:284px" %)**string** 1691 -|(% style="width:366px" %)((( 1692 -**BigInteger** 1965 +)))|**string** 1966 +|((( 1967 +**BigInteger ** 1968 + 1693 1693 (corresponds to XML Schema xs:integer datatype; infinite set of integer values) 1694 -)))|(% style="width:284px" %)**integer** 1695 -|(% style="width:366px" %)((( 1696 -**Integer** 1697 -(corresponds to XML Schema xs:int datatype; between -2147483648 and +2147483647 (inclusive)) 1698 -)))|(% style="width:284px" %)**integer** 1699 -|(% style="width:366px" %)((( 1700 -**Long** 1701 -(corresponds to XML Schema xs:long datatype; between -9223372036854775808 and +9223372036854775807 (inclusive)) 1702 -)))|(% style="width:284px" %)**integer** 1703 -|(% style="width:366px" %)((( 1704 -**Short** 1970 +)))|**integer** 1971 +|((( 1972 +**Integer ** 1973 + 1974 +(corresponds to XML Schema xs:int datatype; between 1975 + 1976 +-2147483648 and +2147483647 (inclusive)) 1977 +)))|**integer** 1978 +|((( 1979 +**Long ** 1980 + 1981 +(corresponds to XML Schema xs:long datatype; 1982 + 1983 +between -9223372036854775808 and +9223372036854775807 (inclusive)) 1984 +)))|**integer** 1985 +|((( 1986 +**Short ** 1987 + 1705 1705 (corresponds to XML Schema xs:short datatype; between -32768 and -32767 (inclusive)) 1706 -)))| (% style="width:284px" %)**integer**1707 -|( % style="width:366px" %)(((1989 +)))|**integer** 1990 +|((( 1708 1708 **Decimal** 1992 + 1709 1709 (corresponds to XML Schema xs:decimal datatype; subset of real numbers that can be represented as decimals) 1710 -)))|(% style="width:284px" %)**number** 1711 -|(% style="width:366px" %)((( 1712 -**Float** 1994 +)))|**number** 1995 +|((( 1996 +**Float ** 1997 + 1713 1713 (corresponds to XML Schema xs:float datatype; patterned after the IEEE single-precision 32-bit floating point type) 1714 -)))|(% style="width:284px" %)**number** 1715 -|(% style="width:366px" %)((( 1716 -**Double** 1999 +)))|**number** 2000 +|((( 2001 +**Double ** 2002 + 1717 1717 (corresponds to XML Schema xs:double datatype; patterned after the IEEE double-precision 64-bit floating point type) 1718 -)))|(% style="width:284px" %)**number** 1719 -|(% style="width:366px" %)((( 1720 -**Boolean** 1721 -(corresponds to the XML Schema xs:boolean datatype; support the mathematical concept of binary-valued logic: {true, false}) 1722 -)))|(% style="width:284px" %)**boolean** 1723 -|(% style="width:366px" %)((( 1724 -**URI** 2004 +)))|**number** 2005 +|((( 2006 +**Boolean ** 2007 + 2008 +(corresponds to the XML Schema xs:boolean datatype; support the mathematical concept of binary-valued logic: {true, false}) 2009 +)))|**boolean** 2010 +|((( 2011 +**URI ** 2012 + 1725 1725 (corresponds to the XML Schema xs:anyURI; absolute or relative Uniform Resource Identifier Reference) 1726 -)))|(% style="width:284px" %)**string** 1727 -|(% style="width:366px" %)((( 1728 -**Count** 2014 +)))|**string** 2015 +|((( 2016 +**Count ** 2017 + 1729 1729 (an integer following a sequential pattern, increasing by 1 for each occurrence) 1730 -)))|(% style="width:284px" %)**integer** 1731 -|(% style="width:366px" %)((( 1732 -**InclusiveValueRange** 2019 +)))|**integer** 2020 +|((( 2021 +**InclusiveValueRange ** 2022 + 1733 1733 (decimal number within a closed interval, whose bounds are specified in the SDMX representation by the facets minValue and maxValue) 1734 -)))|(% style="width:284px" %)**number** 1735 -|(% style="width:366px" %)((( 1736 -**ExclusiveValueRange** 2024 +)))|**number** 2025 +|((( 2026 +**ExclusiveValueRange ** 2027 + 1737 1737 (decimal number within an open interval, whose bounds are specified in the SDMX representation by the facets minValue and maxValue) 1738 -)))|(% style="width:284px" %)**number** 1739 -|(% style="width:366px" %)((( 1740 -**Incremental ** 2029 +)))|**number** 2030 +|((( 2031 +**Incremental ** 2032 + 1741 1741 (decimal number the increased by a specific interval (defined by the interval facet), which is typically enforced outside of the XML validation) 1742 -)))|(% style="width:284px" %)**number** 1743 -|(% style="width:366px" %)((( 1744 -**ObservationalTimePeriod** 2034 +)))|**number** 2035 +|((( 2036 +**ObservationalTimePeriod ** 2037 + 1745 1745 (superset of StandardTimePeriod and TimeRange) 1746 -)))|(% style="width:284px" %)**time** 1747 -|(% style="width:366px" %)((( 1748 -**StandardTimePeriod** 2039 +)))|**time** 2040 +|((( 2041 +**StandardTimePeriod ** 2042 + 1749 1749 (superset of BasicTimePeriod and ReportingTimePeriod) 1750 -)))|(% style="width:284px" %)**time** 1751 -|(% style="width:366px" %)((( 1752 -**BasicTimePeriod** 2044 +)))|**time** 2045 +|((( 2046 +**BasicTimePeriod ** 2047 + 1753 1753 (superset of GregorianTimePeriod and DateTime) 1754 -)))|(% style="width:284px" %)**date** 1755 -|(% style="width:366px" %)((( 1756 -**GregorianTimePeriod** 2049 +)))|**date** 2050 +|((( 2051 +**GregorianTimePeriod ** 2052 + 1757 1757 (superset of GregorianYear, GregorianYearMonth, and GregorianDay) 1758 -)))| (% style="width:284px" %)**date**1759 -| (% style="width:366px" %)**GregorianYear **(YYYY)|(%style="width:284px" %)**date**1760 -| (% style="width:366px" %)**GregorianYearMonth** / **GregorianMonth** (YYYY-MM)|(% style="width:284px" %)**date**1761 -| (% style="width:366px" %)**GregorianDay **(YYYY-MM-DD)|(% style="width:284px" %)**date**1762 -|( % style="width:366px" %)(((2054 +)))|**date** 2055 +|**GregorianYear **(YYYY) |**date** 2056 +|**GregorianYearMonth** / **GregorianMonth** (YYYY-MM)|**date** 2057 +|**GregorianDay **(YYYY-MM-DD)|**date** 2058 +|((( 1763 1763 **ReportingTimePeriod ** 1764 -(superset of RepostingYear, ReportingSemester, ReportingTrimester, ReportingQuarter, ReportingMonth, ReportingWeek, ReportingDay) 1765 -)))|(% style="width:284px" %)**time_period** 1766 -|(% style="width:366px" %)((( 1767 -**ReportingYear** 2060 + 2061 +(superset of RepostingYear, ReportingSemester, 2062 + 2063 +ReportingTrimester, ReportingQuarter, ReportingMonth, 2064 + 2065 +ReportingWeek, ReportingDay) 2066 +)))|**time_period** 2067 +|((( 2068 +**ReportingYear ** 2069 + 1768 1768 (YYYY-A1 – 1 year period) 1769 -)))|(% style="width:284px" %)**time_period** 1770 -|(% style="width:366px" %)((( 1771 -**ReportingSemester** 2071 +)))|**time_period** 2072 +|((( 2073 +**ReportingSemester ** 2074 + 1772 1772 (YYYY-Ss – 6 month period) 1773 -)))|(% style="width:284px" %)**time_period** 1774 -|(% style="width:366px" %)((( 1775 -**ReportingTrimester** 2076 +)))|**time_period** 2077 +|((( 2078 +**ReportingTrimester ** 2079 + 1776 1776 (YYYY-Tt – 4 month period) 1777 -)))|(% style="width:284px" %)**time_period** 1778 -|(% style="width:366px" %)((( 1779 -**ReportingQuarter** 2081 +)))|**time_period** 2082 +|((( 2083 +**ReportingQuarter ** 2084 + 1780 1780 (YYYY-Qq – 3 month period) 1781 -)))|(% style="width:284px" %)**time_period** 1782 -|(% style="width:366px" %)((( 1783 -**ReportingMonth** 2086 +)))|**time_period** 2087 +|((( 2088 +**ReportingMonth ** 2089 + 1784 1784 (YYYY-Mmm – 1 month period) 1785 -)))|(% style="width:284px" %)**time_period** 1786 -|(% style="width:366px" %)((( 1787 -**ReportingWeek** 2091 +)))|**time_period** 2092 +|((( 2093 +**ReportingWeek ** 2094 + 1788 1788 (YYYY-Www – 7 day period; following ISO 8601 definition of a week in a year) 1789 -)))|(% style="width:284px" %)**time_period** 1790 -|(% style="width:366px" %)((( 1791 -**ReportingDay** 2096 +)))|**time_period** 2097 +|((( 2098 +**ReportingDay ** 2099 + 1792 1792 (YYYY-Dddd – 1 day period) 1793 -)))|(% style="width:284px" %)**time_period** 1794 -|(% style="width:366px" %)((( 1795 -**DateTime** 2101 +)))|**time_period** 2102 +|((( 2103 +**DateTime ** 2104 + 1796 1796 (YYYY-MM-DDThh:mm:ss) 1797 -)))| (% style="width:284px" %)**date**1798 -|( % style="width:366px" %)(((1799 -**TimeRange** 2106 +)))|**date** 2107 +|((( 2108 +**TimeRange ** 1800 1800 1801 1801 (YYYY-MM-DD(Thh:mm:ss)?/<duration>) 1802 -)))|(% style="width:284px" %)**time** 1803 -|(% style="width:366px" %)((( 1804 -**Month** 1805 -(~-~-MM; speicifies a month independent of a year; e.g. February is black history month in the United States) 1806 -)))|(% style="width:284px" %)**string** 1807 -|(% style="width:366px" %)((( 1808 -**MonthDay** 1809 -(~-~-MM-DD; specifies a day within a month independent of a year; e.g. Christmas is December 25^^th^^; used to specify reporting year start day) 1810 -)))|(% style="width:284px" %)**string** 1811 -|(% style="width:366px" %)((( 1812 -**Day** 2111 +)))|**time** 2112 +|((( 2113 +**Month ** 2114 + 2115 +(~-~-MM; speicifies a month independent of a year; e.g. 2116 + 2117 +February is black history month in the United States) 2118 +)))|**string** 2119 +|((( 2120 +**MonthDay ** 2121 + 2122 +(~-~-MM-DD; specifies a day within a month independent of a year; e.g. Christmas is December 25^^th^^; used to specify reporting year start day) 2123 +)))|**string** 2124 +|((( 2125 +**Day ** 2126 + 1813 1813 (~-~--DD; specifies a day independent of a month or year; e.g. the 15^^th^^ is payday) 1814 -)))|(% style="width:284px" %)**string** 1815 -|(% style="width:366px" %)((( 1816 -**Time** 2128 +)))|**string** 2129 +|((( 2130 +**Time ** 2131 + 1817 1817 (hh:mm:ss; time independent of a date; e.g. coffee break is at 10:00 AM) 1818 -)))|(% style="width:284px" %)**string** 1819 -|(% style="width:366px" %)((( 1820 -**Duration** 2133 +)))|**string** 2134 +|((( 2135 +**Duration ** 2136 + 1821 1821 (corresponds to XML Schema xs:duration datatype) 1822 -)))| (% style="width:284px" %)**duration**1823 -| (% style="width:366px" %)XHTML|(% style="width:284px" %)Metadata type – not applicable1824 -| (% style="width:366px" %)KeyValues|(% style="width:284px" %)Metadata type – not applicable1825 -| (% style="width:366px" %)IdentifiableReference|(% style="width:284px" %)Metadata type – not applicable1826 -| (% style="width:366px" %)DataSetReference|(% style="width:284px" %)Metadata type – not applicable1827 -| (% style="width:366px" %)AttachmentConstraintReference|(% style="width:284px" %)Metadata type – not applicable2138 +)))|**duration** 2139 +|XHTML|Metadata type – not applicable 2140 +|KeyValues|Metadata type – not applicable 2141 +|IdentifiableReference|Metadata type – not applicable 2142 +|DataSetReference|Metadata type – not applicable 2143 +|AttachmentConstraintReference|Metadata type – not applicable 1828 1828 2145 + 2146 + 1829 1829 **Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types** 1830 1830 1831 1831 When VTL takes in input SDMX artefacts, it is assumed that a type conversion according to the table above always happens. In case a different VTL basic scalar type is desired, it can be achieved in the VTL program taking in input the default VTL basic scalar type above and applying to it the VTL type conversion features (see the implicit and explicit type conversion and the “cast” operator in the VTL Reference Manual). ... ... @@ -1834,84 +1834,89 @@ 1834 1834 1835 1835 The following table describes the default conversion from the VTL basic scalar types to the SDMX data types . 1836 1836 1837 - (%style="width:923.835px"%)1838 -| (% style="width:191px" %)**VTL basic scalartype**|(% style="width:419px"%)**DefaultSDMXdata type(BasicComponentDataType)**|(%style="width:311px" %)**Default output format**1839 -| (% style="width:191px" %)**String**|(% style="width:419px" %)**String**|(% style="width:311px" %)Like XML (xs:string)1840 -| (% style="width:191px" %)**Number**|(% style="width:419px" %)**Float **|(% style="width:311px" %)Like XML (xs:float)1841 -| (% style="width:191px" %)**Integer**|(% style="width:419px" %)**Integer**|(% style="width:311px" %)Like XML (xs:int)1842 -| (% style="width:191px" %)**Date**|(% style="width:419px" %)**DateTime**|(%style="width:311px"%)YYYY-MM-DDT00:00:00Z1843 -| (% style="width:191px" %)**Time**|(% style="width:419px" %)**StandardTimePeriod**|(% style="width:311px" %)<date>/<date>(as defined above)1844 - |(% style="width:191px" %)**time_period**|(% style="width:419px" %)(((1845 - **ReportingTimePeriod1846 -(StandardReportingPeriod)** 1847 -)))|( % style="width:311px" %)(((2155 +|**VTL basic scalar type**|**Default SDMX data type (BasicComponentDataType)**|**Default output format** 2156 +|**String**|**String **|Like XML (xs:string) 2157 +|**Number**|**Float **|Like XML (xs:float) 2158 +|**Integer**|**Integer **|Like XML (xs:int) 2159 +|**Date**|**DateTime**|YYYY-MM-DDT00:00:00Z 2160 +|**Time**|**StandardTimePeriod**|<date>/<date> (as defined above) 2161 +|**time_period**|((( 2162 +**ReportingTimePeriod** 2163 + 2164 +**(StandardReportingPeriod)** 2165 +)))|((( 1848 1848 YYYY-Pppp 2167 + 1849 1849 (according to SDMX ) 1850 1850 ))) 1851 -| (% style="width:191px" %)**Duration**|(% style="width:419px" %)**Duration **|(% style="width:311px" %)(((2170 +|**Duration**|**Duration **|((( 1852 1852 Like XML (xs:duration) 2172 + 1853 1853 PnYnMnDTnHnMnS 1854 1854 ))) 1855 -|(% style="width:191px" %)**Boolean**|(% style="width:419px" %)**Boolean **|(% style="width:311px" %)((( 1856 -Like XML (xs:boolean) with the values “true” or “false” 2175 +|**Boolean**|**Boolean **|((( 2176 +Like XML (xs:boolean) with the values 2177 + 2178 +“true” or “false” 1857 1857 ))) 1858 1858 1859 1859 **Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types** 1860 1860 1861 -In case a different default conversion is desired, it can be achieved through the CustomTypeScheme and CustomType artefacts (see also the section Transformations and Expressions of the SDMX information model).2183 +In case a different default conversion is desired, it can be achieved through the 1862 1862 2185 +CustomTypeScheme and CustomType artefacts (see also the section Transformations and Expressions of the SDMX information model). 2186 + 1863 1863 The custom output formats can be specified by means of the VTL formatting mask described in the section “Type Conversion and Formatting Mask” of the VTL Reference Manual. Such a section describes the masks for the VTL basic scalar types “number”, “integer”, “date”, “time”, “time_period” and “duration” and gives examples. As for the types “string” and “boolean” the VTL conventions are extended with some other special characters as described in the following table. 1864 1864 1865 -(% style="width:671.835px" %) 1866 -|(% colspan="2" style="width:669px" %)**VTL special characters for the formatting masks** 1867 -|(% colspan="2" style="width:669px" %)** ** 1868 -|(% colspan="2" style="width:669px" %)**Number ** 1869 -|(% style="width:141px" %)D|(% style="width:528px" %)one numeric digit (if the scientific notation is adopted, D is only for the mantissa) 1870 -|(% style="width:141px" %)E|(% style="width:528px" %)one numeric digit (for the exponent of the scientific notation) 1871 -|(% style="width:141px" %).(dot)|(% style="width:528px" %)possible separator between the integer and the decimal parts. 1872 -|(% style="width:141px" %),(comma)|(% style="width:528px" %)possible separator between the integer and the decimal parts. 1873 -|(% style="width:141px" %) |(% style="width:528px" %) 1874 -|(% colspan="2" style="width:669px" %)**Time and duration** 1875 -|(% style="width:141px" %)C |(% style="width:528px" %)century 1876 -|(% style="width:141px" %)Y|(% style="width:528px" %)year 1877 -|(% style="width:141px" %)S|(% style="width:528px" %)semester 1878 -|(% style="width:141px" %)Q|(% style="width:528px" %)quarter 1879 -|(% style="width:141px" %)M|(% style="width:528px" %)month 1880 -|(% style="width:141px" %)W|(% style="width:528px" %)week 1881 -|(% style="width:141px" %)D|(% style="width:528px" %)day 1882 -|(% style="width:141px" %)h |(% style="width:528px" %)hour digit (by default on 24 hours) 1883 -|(% style="width:141px" %)M|(% style="width:528px" %)minute 1884 -|(% style="width:141px" %)S|(% style="width:528px" %)second 1885 -|(% style="width:141px" %)D|(% style="width:528px" %)decimal of second 1886 -|(% style="width:141px" %)P|(% style="width:528px" %)period indicator (representation in one digit for the duration) 1887 -|(% style="width:141px" %)P|(% style="width:528px" %)number of the periods specified in the period indicator 1888 -|(% style="width:141px" %)AM/PM |(% style="width:528px" %)indicator of AM / PM (e.g. am/pm for “am” or “pm”) 1889 -|(% style="width:141px" %)MONTH|(% style="width:528px" %)uppercase textual representation of the month (e.g., JANUARY for January) 1890 -|(% style="width:141px" %)DAY|(% style="width:528px" %)uppercase textual representation of the day (e.g., MONDAY for Monday) 1891 -|(% style="width:141px" %)Month|(% style="width:528px" %)lowercase textual representation of the month (e.g., january) 1892 -|(% style="width:141px" %)Day|(% style="width:528px" %)lowercase textual representation of the month (e.g., monday) 1893 -|(% style="width:141px" %)Month|(% style="width:528px" %)First character uppercase, then lowercase textual representation of the month (e.g., January) 1894 -|(% style="width:141px" %)Day|(% style="width:528px" %)First character uppercase, then lowercase textual representation of the day using (e.g. Monday) 1895 -|(% style="width:141px" %) |(% style="width:528px" %) 1896 -|(% colspan="2" style="width:669px" %)**String** 1897 -|(% style="width:141px" %)X|(% style="width:528px" %)any string character 1898 -|(% style="width:141px" %)Z|(% style="width:528px" %)any string character from “A” to “z” 1899 -|(% style="width:141px" %)9|(% style="width:528px" %)any string character from “0” to “9” 1900 -|(% style="width:141px" %) |(% style="width:528px" %) 1901 -|(% colspan="2" style="width:669px" %)**Boolean ** 1902 -|(% style="width:141px" %)B|(% style="width:528px" %)Boolean using “true” for True and “false” for False 1903 -|(% style="width:141px" %)1|(% style="width:528px" %)Boolean using “1” for True and “0” for False 1904 -|(% style="width:141px" %)0|(% style="width:528px" %)Boolean using “0” for True and “1” for False 1905 -|(% style="width:141px" %) |(% style="width:528px" %) 1906 -|(% colspan="2" style="width:669px" %)Other qualifiers 1907 -|(% style="width:141px" %)*|(% style="width:528px" %)an arbitrary number of digits (of the preceding type) 1908 -|(% style="width:141px" %)+|(% style="width:528px" %)at least one digit (of the preceding type) 1909 -|(% style="width:141px" %)( )|(% style="width:528px" %)optional digits (specified within the brackets) 1910 -|(% style="width:141px" %)\|(% style="width:528px" %)prefix for the special characters that must appear in the mask 1911 -|(% style="width:141px" %)N|(% style="width:528px" %)fixed number of digits used in the preceding textual representation of the month or the day 1912 -|(% style="width:141px" %) |(% style="width:528px" %) 2189 +|(% colspan="2" %)**VTL special characters for the formatting masks** 2190 +|(% colspan="2" %)** ** 2191 +|(% colspan="2" %)**Number ** 2192 +|D|one numeric digit (if the scientific notation is adopted, D is only for the mantissa) 2193 +|E|one numeric digit (for the exponent of the scientific notation) 2194 +|. (dot)|possible separator between the integer and the decimal parts. 2195 +|, (comma)|possible separator between the integer and the decimal parts. 2196 +| | 2197 +|(% colspan="2" %)**Time and duration** 2198 +|C |century 2199 +|Y|year 2200 +|S|semester 2201 +|Q|quarter 2202 +|M|month 2203 +|W|week 2204 +|D|day 2205 +|h |hour digit (by default on 24 hours) 2206 +|M|minute 2207 +|S|second 2208 +|D|decimal of second 2209 +|P|period indicator (representation in one digit for the duration) 2210 +|P|number of the periods specified in the period indicator 2211 +|AM/PM |indicator of AM / PM (e.g. am/pm for “am” or “pm”) 2212 +|MONTH|uppercase textual representation of the month (e.g., JANUARY for January) 2213 +|DAY|uppercase textual representation of the day (e.g., MONDAY for Monday) 2214 +|Month|lowercase textual representation of the month (e.g., january) 2215 +|Day|lowercase textual representation of the month (e.g., monday) 2216 +|Month|First character uppercase, then lowercase textual representation of the month (e.g., January) 2217 +|Day|First character uppercase, then lowercase textual representation of the day using (e.g. Monday) 2218 +| | 2219 +|(% colspan="2" %)**String ** 2220 +|X|any string character 2221 +|Z|any string character from “A” to “z” 2222 +|9|any string character from “0” to “9” 2223 +| | 2224 +|(% colspan="2" %)**Boolean ** 2225 +|B|Boolean using “true” for True and “false” for False 2226 +|1|Boolean using “1” for True and “0” for False 2227 +|0|Boolean using “0” for True and “1” for False 2228 +| | 2229 +|(% colspan="2" %)Other qualifiers 2230 +|*|an arbitrary number of digits (of the preceding type) 2231 +|+|at least one digit (of the preceding type) 2232 +|( )|optional digits (specified within the brackets) 2233 +|\|prefix for the special characters that must appear in the mask 2234 +|N|fixed number of digits used in the preceding textual representation of the month or the day 2235 +| | 1913 1913 1914 -The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[43~]^^>>path:#_ftn43]](%%).2237 +The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion[[(% class="wikiinternallink wikiinternallink" %)^^~[43~]^^>>path:#_ftn43]](%%). 1915 1915 1916 1916 === 10.4.5 Null Values === 1917 1917 ... ... @@ -1919,20 +1919,22 @@ 1919 1919 1920 1920 On the other side, the VTL programs can produce in output NULL values for Measures and Attributes (Null values are not allowed in the Identifiers). In the conversion from VTL to SDMX, it is assumed that a NULL in VTL becomes a missing value in SDMX. 1921 1921 1922 -In the conversion from VTL to SDMX, the default assumption can be overridden, separately for each VTL basic scalar type, by specifying which the value that represents the NULL in SDMX is. This can be specified in the attribute “nullValue” of the CustomType artefact (see also the section Transformations and Expressions of the SDMX information model). A CustomType belongs to a CustomTypeScheme, which can be referenced by one or more TransformationScheme (i.e. VTL programs). The overriding assumption is applied for all the SDMX Dataflows calculated in the TransformationScheme. 2245 +In the conversion from VTL to SDMX, the default assumption can be overridden, separately for each VTL basic scalar type, by specifying which the value that represents the NULL in SDMX is. This can be specified in the attribute “nullValue” of the CustomType artefact (see also the section Transformations and Expressions of the SDMX information model). A CustomType belongs to a CustomTypeScheme, which can be referenced by one or more TransformationScheme (i.e. VTL programs). The overriding assumption is applied for all the SDMX Dataflows calculated in the TransformationScheme. 1923 1923 1924 1924 === 10.4.6 Format of the literals used in VTL transformations === 1925 1925 1926 1926 The VTL programs can contain literals, i.e. specific values of certain data types written directly in the VTL definitions or expressions. The VTL does not prescribe a specific format for the literals and leave the specific VTL systems and the definers of VTL transformations free of using their preferred formats. 1927 1927 1928 -Given this discretion, it is essential to know which are the external representations adopted for the literals in a VTL program, in order to interpret them correctly. For example, if the external format for the dates is YYYY-MM-DD the date literal 201001-02 has the meaning of 2^^nd^^ January 2010, instead if the external format for the dates is YYYY-DD-MM the same literal has the meaning of 1^^st^^ February 2010. 2251 +Given this discretion, it is essential to know which are the external representations adopted for the literals in a VTL program, in order to interpret them correctly. For example, if the external format for the dates is YYYY-MM-DD the date literal 201001-02 has the meaning of 2^^nd^^ January 2010, instead if the external format for the dates is YYYY-DD-MM the same literal has the meaning of 1^^st^^ February 2010. 1929 1929 1930 1930 Hereinafter, i.e. in the SDMX implementation of the VTL, it is assumed that the literals are expressed according to the “default output format” of the table of the previous paragraph (“Mapping VTL basic scalar types to SDMX data types”) unless otherwise specified. 1931 1931 1932 1932 A different format can be specified in the attribute “vtlLiteralFormat” of the CustomType artefact (see also the section Transformations and Expressions of the SDMX information model). 1933 1933 1934 -Like in the case of the conversion of NULLs described in the previous paragraph, the overriding assumption is applied, for a certain VTL basic scalar type, if a value is found for the vtlLiteralFormat attribute of the CustomType of such VTL basic scalar type. The overriding assumption is applied for all the literals of a related VTL TransformationScheme.2257 +Like in the case of the conversion of NULLs described in the previous paragraph, the overriding assumption is applied, for a certain VTL basic scalar type, if a value is found for the vtlLiteralFormat attribute of the CustomType of such VTL basic scalar type. The overriding assumption is applied for all the literals of a related VTL 1935 1935 2259 +TransformationScheme. 2260 + 1936 1936 In case a literal is operand of a VTL Cast operation, the format specified in the Cast overrides all the possible otherwise specified formats. 1937 1937 1938 1938 = 11 Annex I: How to eliminate extra element in the .NET SDMX Web Service = ... ... @@ -1941,18 +1941,12 @@ 1941 1941 1942 1942 For implementing an SDMX compliant Web Service the standardised WSDL file should be used that describes the expected request/response structure. The request message of the operation contains a wrapper element (e.g. “GetGenericData”) that wraps a tag called “GenericDataQuery”, which is the actual SDMX query XML message that contains the query to be processed by the Web Service. In the same way the response is formulated in a wrapper element “GetGenericDataResponse”. 1943 1943 1944 -As defined in the SOAP specification, the root element of a SOAP message is the Envelope, which contains an optional Header and a mandatory Body. These are illustrated below along with the Body contents according to the WSDL: 2269 +As defined in the SOAP specification, the root element of a SOAP message is the Envelope, which contains an optional Header and a mandatory Body. These are illustrated below along with the Body contents according to the WSDL: 1945 1945 1946 -[[image:1747854006117-843.png]] 1947 - 1948 1948 The problem that initiated the present analysis refers to the difference in the way SOAP requests are when trying to implement the aforementioned Web Service in .NET framework. 1949 1949 1950 1950 Building such a Web Service using the .NET framework is done by exposing a method (i.e. the getGenericData in the example) with an XML document argument (lets name it “Query”). **The difference that appears in Microsoft .Net implementations is that there is a need for an extra XML container around the SDMX GenericDataQuery.** This is the expected behavior since the framework is let to publish automatically the Web Service as a remote procedure call, thus wraps each parameter into an extra element. The .NET request is illustrated below: 1951 1951 1952 -[[image:1747854039499-443.png]] 1953 - 1954 -[[image:1747854067769-691.png]] 1955 - 1956 1956 Furthermore this extra element is also inserted in the automatically generated WSDL from the framework. Therefore this particularity requires custom clients for the .NET Web Services that is not an interoperable solution. 1957 1957 1958 1958 == 11.2 Solution == ... ... @@ -1973,30 +1973,20 @@ 1973 1973 1974 1974 To understand how the **XmlAnyElement** attribute works we present the following two web methods: 1975 1975 1976 - [[image:1747854096778-844.png]]2295 +In this method the **input** parameter is decorated with the **XmlAnyElement** parameter. This is a hint that this parameter will be de-serialized from an **xsd:any** element. Since the attribute is not passed any parameters, it means that the entire XML element for this parameter in the SOAP message will be in the Infoset that is represented by this **XmlElement** parameter. 1977 1977 1978 - In this methodthe **input** parameteris decoratedwith the**XmlAnyElement** parameter. This is a hint that this parameterwill bede-serialized from an**xsd:any** element. Since theattribute is notpassed any parameters,it means thatthe entire XML elementfor this parameterintheSOAPmessage will be inthe Infoset that is representedby this**XmlElement**parameter.2297 +The difference between the two is that for the first method, **SubmitXml**, the 1979 1979 1980 - [[image:1747854127303-270.png]]2299 +XmlSerializer will expect an element named **input** to be an immediate child of the **SubmitXml** element in the SOAP body. The second method, **SubmitXmlAny**, will not care what the name of the child of the **SubmitXmlAny** element is. It will plug whatever XML is included into the input parameter. The message style from ASP.NET Help for the two methods is shown below. First we look at the message for the method without the **XmlAnyElement** attribute. 1981 1981 1982 -The difference between the two is that for the first method, **SubmitXml**, the XmlSerializer will expect an element named **input** to be an immediate child of the **SubmitXml** element in the SOAP body. The second method, **SubmitXmlAny**, will not care what the name of the child of the **SubmitXmlAny** element is. It will plug whatever XML is included into the input parameter. The message style from ASP.NET Help for the two methods is shown below. First we look at the message for the method without the **XmlAnyElement** attribute. 1983 - 1984 -[[image:1747854163928-581.png]] 1985 - 1986 1986 Now we look at the message for the method that uses the **XmlAnyElement** attribute. 1987 1987 1988 -[[image:1747854190641-364.png]] 1989 - 1990 -[[image:1747854236732-512.png]] 1991 - 1992 1992 The method decorated with the **XmlAnyElement** attribute has one fewer wrapping elements. Only an element with the name of the method wraps what is passed to the **input** parameter. 1993 1993 1994 -For more information please consult: [[http:~~/~~/msdn.microsoft.com/en-us/library/aa480498.aspx>>http://msdn.microsoft.com/en-us/library/aa480498.aspx]] 2305 +For more information please consult: [[http:~~/~~/msdn.microsoft.com/en>>url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]][[->>url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]][[us/library/aa480498.aspx>>url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]][[url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]] 1995 1995 1996 1996 Furthermore at this point the problem with the different requests has been solved. However there is still the difference in the produced WSDL that has to be taken care. The automatic generated WSDL now doesn’t insert the extra element, but defines the content of the operation wrapper element as “xsd:any” type. 1997 1997 1998 -[[image:1747854286398-614.png]] 1999 - 2000 2000 Without a common WSDL still the solution doesn’t enforce interoperability. In order to 2001 2001 2002 2002 “fix” the WSDL, there two approaches. The first is to intervene in the generation process. This is a complicated approach, compared to the second approach, which overrides the generation process and returns the envisioned WSDL for the SDMX Web Service. ... ... @@ -2009,27 +2009,16 @@ 2009 2009 2010 2010 In the context of the SDMX Web Service, applying the above solution translates into the following: 2011 2011 2012 -[[image:1747854385465-132.png]] 2013 - 2014 2014 The SOAP request/response will then be as follows: 2015 2015 2016 2016 **GenericData Request** 2017 2017 2018 -[[image:1747854406014-782.png]] 2019 - 2020 2020 **GenericData Response** 2021 2021 2022 -[[image:1747854424488-855.png]] 2023 - 2024 2024 For overriding the automatically produced WSDL, in the solution explorer right click the project and select “Add” -> “New item…”. Then select the “Global Application Class”. This will create “.asax” class file in which the following code should replace the existing empty method: 2025 2025 2026 -[[image:1747854453895-524.png]] 2027 - 2028 -[[image:1747854476631-125.png]] 2029 - 2030 2030 The SDMX_WSDL.wsdl should reside in the in the root directory of the application. After applying this solution the returned WSDL is the envisioned. Thus in the request message definition contains: 2031 2031 2032 -[[image:1747854493363-776.png]] 2033 2033 2034 2034 ---- 2035 2035 ... ... @@ -2057,15 +2057,15 @@ 2057 2057 2058 2058 [[~[12~]>>path:#_ftnref12]] In case the invoked artefact is a VTL component, which can be invoked only within the invocation of a 2059 2059 2060 -VTL data set (SDMX dataflow), the specific SDMX class-name (e.g. Dimension, MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute) can be deduced from the data structure of the SDMX Dataflow which the component belongs to. 2358 +VTL data set (SDMX dataflow), the specific SDMX class-name (e.g. Dimension, MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute) can be deduced from the data structure of the SDMX Dataflow which the component belongs to. 2061 2061 2062 -[[~[13~]>>path:#_ftnref13]] If the Agency is composite (for example AgencyA.Dept1.Unit2), the agency is considered different even if only part of the composite name is different (for example AgencyA.Dept1.Unit3 is a different Agency than the previous one). Moreover the agency-id cannot be omitted in part (i.e., if a TransformationScheme owned by AgencyA.Dept1.Unit2 references an artefact coming from AgencyA.Dept1.Unit3, the specification of the agency-id becomes mandatory and must be complete, without omitting the possibly equal parts like AgencyA.Dept1) 2360 +[[~[13~]>>path:#_ftnref13]] If the Agency is composite (for example AgencyA.Dept1.Unit2), the agency is considered different even if only part of the composite name is different (for example AgencyA.Dept1.Unit3 is a different Agency than the previous one). Moreover the agency-id cannot be omitted in part (i.e., if a TransformationScheme owned by AgencyA.Dept1.Unit2 references an artefact coming from AgencyA.Dept1.Unit3, the specification of the agency-id becomes mandatory and must be complete, without omitting the possibly equal parts like AgencyA.Dept1) 2063 2063 2064 2064 [[~[14~]>>path:#_ftnref14]] Single quotes are needed because this reference is not a VTL regular name. 2065 2065 2066 2066 [[~[15~]>>path:#_ftnref15]] Single quotes are not needed in this case because CL_FREQ is a VTL regular name. 2067 2067 2068 -[[~[16~]>>path:#_ftnref16]] The result DFR(1.0) is be equal to DF1(1.0) save that the component SECTOR is called SEC 2366 +[[~[16~]>>path:#_ftnref16]] The result DFR(1.0) is be equal to DF1(1.0) save that the component SECTOR is called SEC 2069 2069 2070 2070 [[~[17~]>>path:#_ftnref17]] Rulesets of this kind cannot be reused when the referenced Concept has a different representation. 2071 2071 ... ... @@ -2081,7 +2081,7 @@ 2081 2081 2082 2082 [[~[23~]>>path:#_ftnref23]] The SDMX community is evaluating the opportunity of allowing more than one measure component in a DataStructureDefinition in the next SDMX major version. 2083 2083 2084 -[[~[24~]>>path:#_ftnref24]] If future SDMX major versions will allow multi-measures data structures, this method is expected to become applicable even if the VTL data structure has more than one measure 2382 +[[~[24~]>>path:#_ftnref24]] If future SDMX major versions will allow multi-measures data structures, this method is expected to become applicable even if the VTL data structure has more than one measure 2085 2085 2086 2086 [[~[25~]>>path:#_ftnref25]] The kind of mapping explained here works in combination with a SDMX specific naming convention that requires pre-processing before parsing the VTL expressions. As highlighted below, the identifiers of the VTL datasets are a shortcut of some specific VTL operators applied to the SDMX Dataflows. This is not safe to use outside an SDMX context, as the naming convention may have no meaning there. 2087 2087 ... ... @@ -2089,7 +2089,7 @@ 2089 2089 2090 2090 [[~[27~]>>path:#_ftnref27]] Please note that this kind of mapping is only an option at disposal of the definer of VTL Transformations; in fact it remains always possible to manipulate the needed parts of SDMX Dataflows by means of VTL operators (e.g. “sub”, “filter”, “calc”, “union” …), maintaining a mapping one-to-one between SDMX Dataflows and VTL datasets. 2091 2091 2092 -[[~[28~]>>path:#_ftnref28]] This definition is made through the ToVtlSubspace and ToVtlSpaceKey classes and/or the FromVtlSuperspace and FromVtlSpaceKey classes, depending on the direction of the mapping (“key” means “dimension”). The mapping of Dataflow subsets can be applied independently in the two directions, also according to different Dimensions. When no Dimension is declared for a given direction, it is assumed that the option of mapping different parts of a SDMX Dataflow to different VTL datasets is not used. 2390 +[[~[28~]>>path:#_ftnref28]] This definition is made through the ToVtlSubspace and ToVtlSpaceKey classes and/or the FromVtlSuperspace and FromVtlSpaceKey classes, depending on the direction of the mapping (“key” means “dimension”). The mapping of Dataflow subsets can be applied independently in the two directions, also according to different Dimensions. When no Dimension is declared for a given direction, it is assumed that the option of mapping different parts of a SDMX Dataflow to different VTL datasets is not used. 2093 2093 2094 2094 [[~[29~]>>path:#_ftnref29]] As a consequence of this formalism, a slash in the name of the VTL dataset assumes the specific meaning of separator between the name of the Dataflow and the values of some of its Dimensions. 2095 2095 ... ... @@ -2097,13 +2097,13 @@ 2097 2097 2098 2098 [[~[31~]>>path:#_ftnref31]] It should be remembered that, according to the VTL consistency rules, a given VTL dataset cannot be the result of more than one VTL transformation. 2099 2099 2100 -[[~[32~]>>path:#_ftnref32]] If these dimensions would not be dropped, taking into account that the typical binary VTL operations at dataset level (+, -, *, / and so on) are executed on the observations having matching identifiers, the VTL datasets resulting from this kind of mapping would have non-matching values for the mapping dimensions (e.g. POPULATION and COUNTRY), therefore it would not be possible to compose the resulting VTL datasets one another (e.g. it would not be possible to calculate the population ratio between USA and CANADA). ^^ ^^ 2398 +[[~[32~]>>path:#_ftnref32]] If these dimensions would not be dropped, taking into account that the typical binary VTL operations at dataset level (+, -, *, / and so on) are executed on the observations having matching identifiers, the VTL datasets resulting from this kind of mapping would have non-matching values for the mapping dimensions (e.g. POPULATION and COUNTRY), therefore it would not be possible to compose the resulting VTL datasets one another (e.g. it would not be possible to calculate the population ratio between USA and CANADA). ^^ ^^ 2101 2101 2102 -[[~[33~]>>path:#_ftnref33]] In case the ordered concatenation notation is used, the VTL Transformation described above, e.g. 2400 +[[~[33~]>>path:#_ftnref33]] In case the ordered concatenation notation is used, the VTL Transformation described above, e.g. 2103 2103 2104 -‘DF1(1.0)/POPULATION.USA’ := DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA”], is implicitly executed and, in order to test the overall compliance of the VTL program to the VTL consistency rules, it has to be considered as part of the VTL program even if it is not explicitly coded. 2402 +‘DF1(1.0)/POPULATION.USA’ := DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA”], is implicitly executed and, in order to test the overall compliance of the VTL program to the VTL consistency rules, it has to be considered as part of the VTL program even if it is not explicitly coded. 2105 2105 2106 -[[~[34~]>>path:#_ftnref34]] If the whole DF2(1.0) is calculated by means of just one VTL transformation, then the mapping between the SDMX dataflow and the corresponding VTL dataset is one-to-one and this kind of mapping (one SDMX Dataflow to many VTL datasets) does not apply.. 2404 +[[~[34~]>>path:#_ftnref34]] If the whole DF2(1.0) is calculated by means of just one VTL transformation, then the mapping between the SDMX dataflow and the corresponding VTL dataset is one-to-one and this kind of mapping (one SDMX Dataflow to many VTL datasets) does not apply.. 2107 2107 2108 2108 [[~[35~]>>path:#_ftnref35]] This is possible as each VTL dataset corresponds to one particular combination of values of INDICATOR and COUNTRY 2109 2109 ... ... @@ -2122,5 +2122,3 @@ 2122 2122 [[~[42~]>>path:#_ftnref42]] A Concept becomes a Component in a DataStructureDefinition, and Components can have different LocalRepresentations in different DataStructureDefinitions, also overriding the (possible) base representation of the Concept. 2123 2123 2124 2124 [[~[43~]>>path:#_ftnref43]] The representation given in the DSD should obviously be compatible with the VTL data type. 2125 - 2126 -{{putFootnotes/}}
- 1747854006117-843.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -18.7 KB - Content
- 1747854039499-443.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -3.0 KB - Content
- 1747854067769-691.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -21.4 KB - Content
- 1747854096778-844.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -10.6 KB - Content
- 1747854127303-270.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -12.1 KB - Content
- 1747854163928-581.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -29.8 KB - Content
- 1747854190641-364.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -31.5 KB - Content
- 1747854236732-512.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -2.0 KB - Content
- 1747854286398-614.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747854385465-132.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747854406014-782.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747854424488-855.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747854453895-524.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747854476631-125.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747854493363-776.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747855024745-946.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747855054559-410.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747855075263-887.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747855402867-579.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747855462293-368.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747855493531-357.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747856459132-767.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747859398930-848.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747859458410-183.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747859612718-454.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -55.0 KB - Content
- 1747859722732-549.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -18.8 KB - Content