Changes for page SDMX 2.1 Standards. Section 6. Technical Notes
Last modified by Artur on 2025/08/19 10:43
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 15 added, 0 removed)
- 1747854006117-843.png
- 1747854039499-443.png
- 1747854067769-691.png
- 1747854096778-844.png
- 1747854127303-270.png
- 1747854163928-581.png
- 1747854190641-364.png
- 1747854236732-512.png
- 1747854286398-614.png
- 1747854385465-132.png
- 1747854406014-782.png
- 1747854424488-855.png
- 1747854453895-524.png
- 1747854476631-125.png
- 1747854493363-776.png
Details
- Page properties
-
- Content
-
... ... @@ -1,5 +1,9 @@ 1 -Revision History 1 +{{box title="**Contents**"}} 2 +{{toc/}} 3 +{{/box}} 2 2 5 +**Revision History** 6 + 3 3 |**Revision**|**Date**|**Contents** 4 4 | |April 2011|Initial release 5 5 |1.0|April 2013|Added section 9 - Transforming between versions of SDMX ... ... @@ -9,10 +9,8 @@ 9 9 10 10 == 1.1 Purpose == 11 11 12 -The intention of this document is to document certain aspects of SDMX that are important to understand and will aid implementation decisions. The explanations here supplement the information documented in the SDMX XML schema and the 16 +The intention of this document is to document certain aspects of SDMX that are important to understand and will aid implementation decisions. The explanations here supplement the information documented in the SDMX XML schema and the Information Model. 13 13 14 -Information Model. 15 - 16 16 == 1.2 Structure == 17 17 18 18 This document is organized into the following major parts: ... ... @@ -37,7 +37,7 @@ 37 37 38 38 == 3.2 SDMX Information Model for Format Implementers == 39 39 40 -=== 3.2.1 42 +=== 3.2.1 Introduction === 41 41 42 42 The purpose of this sub-section is to provide an introduction to the SDMX-IM relating to Data Structure Definitions and Data Sets for those whose primary interest is in the use of the XML or EDI formats. For those wishing to have a deeper understanding of the Information Model, the full SDMX-IM document, and other sections in this guide provide a more in-depth view, along with UML diagrams and supporting explanation. For those who are unfamiliar with DSDs, an appendix to the SDMX-IM provides a tutorial which may serve as a useful introduction. 43 43 ... ... @@ -45,16 +45,12 @@ 45 45 46 46 The Data Structure Definition and Data Set parts of the information model are consistent with the GESMES/TS version 3.0 Data Model (called SDMX-EDI in the SDMX standard), with these exceptions: 47 47 48 -the “sibling group” construct has been generalized to permit any dimension or dimensions to be wildcarded, and not just frequency, as in GESMES/TS. It has been renamed a “group” to distinguish it from the “sibling group” where only frequency is wildcarded. The set of allowable partial “group” keys must be declared in the DSD, and attributes may be attached to any of these group keys; 50 +* the “sibling group” construct has been generalized to permit any dimension or dimensions to be wildcarded, and not just frequency, as in GESMES/TS. It has been renamed a “group” to distinguish it from the “sibling group” where only frequency is wildcarded. The set of allowable partial “group” keys must be declared in the DSD, and attributes may be attached to any of these group keys; 51 +* furthermore, whilst the “group” has been retained for compatibility with version 2.0 and with SDMX-EDI, it has, at version 2.1, been replaced by the “Attribute Relationship” definition which is explained later 52 +* the section on data representation is now a convention, to support interoperability with EDIFACT-syntax implementations ( see section 3.3.2); 49 49 50 -f urthermore,whilstthe“group”hasbeenretainedfor compatibility withversion2.0andwithSDMX-EDI, itas,atversion2.1,beenreplacedby the“AttributeRelationship”definitionwhichis explainedlater54 +DSD-specific data formats are derived from the model, and some supporting features for declaring multiple measures have been added to the structural metadata descriptions Clearly, this is not a coincidence. The GESMES/TS Data Model provides the foundation for the EDIFACT messages in SDMX-EDI, and also is the starting point for the development of SDMX-ML. 51 51 52 -the section on data representation is now a convention, to support interoperability with EDIFACT-syntax implementations ( see section 3.3.2); 53 - 54 -DSD-specific data formats are derived from the model, and some supporting features for declaring multiple measures have been added to the structural metadata descriptions 55 - 56 -Clearly, this is not a coincidence. The GESMES/TS Data Model provides the foundation for the EDIFACT messages in SDMX-EDI, and also is the starting point for the development of SDMX-ML. 57 - 58 58 Note that in the descriptions below, text in courier and italicised are the names used in the information model (e.g. //DataSet//). 59 59 60 60 == 3.3 SDMX-ML and SDMX-EDI: Comparison of Expressive Capabilities and Function == ... ... @@ -61,22 +61,16 @@ 61 61 62 62 SDMX offers several equivalent formats for describing data and structural metadata, optimized for use in different applications. Although all of these formats are derived directly from the SDM-IM, and are thus equivalent, the syntaxes used to express the model place some restrictions on their use. Also, different optimizations provide different capabilities. This section describes these differences, and provides some rules for applications which may need to support more than one SDMX format or syntax. This section is constrained to the Data Structure Definitionand the Date Set. 63 63 64 -=== 3.3.1 62 +=== 3.3.1 Format Optimizations and Differences === 65 65 66 66 The following section provides a brief overview of the differences between the various SDMX formats. 67 67 68 -Version 2.0 was characterised by 4 data messages, each with a distinct format: Generic, Compact, Cross-Sectional and Utility. Because of the design, data in some formats could not always be related to another format. In version 2.1, this issue has been addressed by merging some formats and eliminating others. As a result, in 66 +Version 2.0 was characterised by 4 data messages, each with a distinct format: Generic, Compact, Cross-Sectional and Utility. Because of the design, data in some formats could not always be related to another format. In version 2.1, this issue has been addressed by merging some formats and eliminating others. As a result, in SDMX 2.1 there are just two types of data formats: //GenericData// and //StructureSpecificData// (i.e. specific to one Data Structure Definition). 69 69 70 -SDMX 2.1 there are just two types of data formats: //GenericData// and 71 - 72 -//StructureSpecificData// (i.e. specific to one Data Structure Definition). 73 - 74 74 Both of these formats are now flexible enough to allow for data to be oriented in series with any dimension used to disambiguate the observations (as opposed to only time or a cross sectional measure in version 2.0). The formats have also been expanded to allow for ungrouped observations. 75 75 76 -To allow for applications which only understand time series data, variations of these formats have been introduced in the form of two data messages; 70 +To allow for applications which only understand time series data, variations of these formats have been introduced in the form of two data messages; //GenericTimeSeriesData// and //StructureSpecificTimeSeriesData//. It is important to note that these variations are built on the same root structure and can be processed in the same manner as the base format so that they do NOT introduce additional processing requirements. 77 77 78 -//GenericTimeSeriesData// and //StructureSpecificTimeSeriesData//. It is important to note that these variations are built on the same root structure and can be processed in the same manner as the base format so that they do NOT introduce additional processing requirements. 79 - 80 80 === //Structure Definition// === 81 81 82 82 The SDMX-ML Structure Message supports the use of annotations to the structure, which is not supported by the SDMX-EDI syntax. ... ... @@ -85,10 +85,8 @@ 85 85 86 86 === //Validation// === 87 87 88 -SDMX-EDI – as is typical of EDIFACT syntax messages – leaves validation to dedicated applications (“validation” being the checking of syntax, data typing, and adherence of the data message to the structure as described in the structural 80 +SDMX-EDI – as is typical of EDIFACT syntax messages – leaves validation to dedicated applications (“validation” being the checking of syntax, data typing, and adherence of the data message to the structure as described in the structural definition.) 89 89 90 -definition.) 91 - 92 92 The SDMX-ML Generic Data Message also leaves validation above the XML syntax level to the application. 93 93 94 94 The SDMX-ML DSD-specific messages will allow validation of XML syntax and datatyping to be performed with a generic XML parser, and enforce agreement between the structural definition and the data to a moderate degree with the same tool. ... ... @@ -99,17 +99,13 @@ 99 99 100 100 === //Character Encodings// === 101 101 102 -All SDMX-ML messages use the UTF-8 encoding, while SDMX-EDI uses the ISO 8879-1 character encoding. There is a greater capacity with UTF-8 to express some character sets (see the “APPENDIX: MAP OF ISO 8859-1 (UNOC) CHARACTER 92 +All SDMX-ML messages use the UTF-8 encoding, while SDMX-EDI uses the ISO 8879-1 character encoding. There is a greater capacity with UTF-8 to express some character sets (see the “APPENDIX: MAP OF ISO 8859-1 (UNOC) CHARACTER SET (LATIN 1 OR “WESTERN”) in the document “SYNTAX AND DOCUMENTATION VERSION 2.0”.) Many transformation tools are available which allow XML instances with UTF-8 encodings to be expressed as ISO 8879-1-encoded characters, and to transform UTF-8 into ISO 8879-1. Such tools should be used when transforming SDMX-ML messages into SDMX-EDI messages and vice-versa. 103 103 104 -SET (LATIN 1 OR “WESTERN”) in the document “SYNTAX AND 105 - 106 -DOCUMENTATION VERSION 2.0”.) Many transformation tools are available which allow XML instances with UTF-8 encodings to be expressed as ISO 8879-1-encoded characters, and to transform UTF-8 into ISO 8879-1. Such tools should be used when transforming SDMX-ML messages into SDMX-EDI messages and vice-versa. 107 - 108 108 === //Data Typing// === 109 109 110 110 The XML syntax and EDIFACT syntax have different data-typing mechanisms. The section below provides a set of conventions to be observed when support for messages in both syntaxes is required. For more information on the SDMX-ML representations of data, see below. 111 111 112 -==== 3.3.2 98 +==== 3.3.2 Data Types ==== 113 113 114 114 The XML syntax has a very different mechanism for data-typing than the EDIFACT syntax, and this difference may create some difficulties for applications which support both EDIFACT-based and XML-based SDMX data formats. This section provides a set of conventions for the expression in data in all formats, to allow for clean interoperability between them. 115 115 ... ... @@ -125,7 +125,8 @@ 125 125 1*. Maximum 70 characters. 126 126 1*. From ISO 8859-1 character set (including accented characters) 127 127 1. **Descriptions **are: 128 -1*. Maximum 350 characters; From ISO 8859-1 character set. 114 +1*. Maximum 350 characters; 115 +1*. From ISO 8859-1 character set. 129 129 1. **Code values** are: 130 130 1*. Maximum 18 characters; 131 131 1*. Any of A..Z (upper case alphabetic), 0..9 (numeric), _ (underscore), / (solidus, slash), = (equal sign), - (hyphen); ... ... @@ -134,37 +134,43 @@ 134 134 135 135 A..Z (upper case alphabetic), 0..9 (numeric), _ (underscore) 136 136 137 -1. **Observation values** are: 138 -1*. Decimal numerics (signed only if they are negative); 139 -1*. The maximum number of significant figures is: 140 -1*. 15 for a positive number 141 -1*. 14 for a positive decimal or a negative integer 142 -1*. 13 for a negative decimal 143 -1*. Scientific notation may be used. 144 -1. **Uncoded statistical concept** text values are: 145 -1*. 146 -1**. Maximum 1050 characters; 147 -1**. From ISO 8859-1 character set. 148 -1. **Time series keys**: 124 +**5. Observation values** are: 149 149 150 -In principle, the maximum permissible length of time series keys used in a data exchange does not need to be restricted. However, for working purposes, an effort is made to limit the maximum length to 35 characters; in this length, also (for SDMXEDI) one (separator) position is included between all successive dimension values; this means that the maximum length allowed for a pure series key (concatenation of dimension values) can be less than 35 characters. The separator character is a colon (“:”) by conventional usage. 126 +* Decimal numerics (signed only if they are negative); 127 +* The maximum number of significant figures is: 128 +* 15 for a positive number 129 +* 14 for a positive decimal or a negative integer 130 +* 13 for a negative decimal 131 +* Scientific notation may be used. 151 151 133 +**6. Uncoded statistical concept** text values are: 134 + 135 +* Maximum 1050 characters; 136 +* From ISO 8859-1 character set. 137 + 138 +**7. Time series keys**: 139 + 140 +In principle, the maximum permissible length of time series keys used in a data exchange does not need to be restricted. However, for working purposes, an effort is made to limit the maximum length to 35 characters; in this length, also (for SDMXEDI) one (separator) position is included between all successive dimension values; this means that the maximum length allowed for a pure series key (concatenation of dimension values) can be less than 35 characters. The separator character is a colon (“:”) by conventional usage. 141 + 152 152 == 3.4 SDMX-ML and SDMX-EDI Best Practices == 153 153 154 -=== 3.4.1 144 +=== 3.4.1 Reporting and Dissemination Guidelines === 155 155 156 - **3.4.1.1 Central Institutions and Their Role in Statistical Data Exchanges**Central institutions are the organisations to which other partner institutions "report" statistics. These statistics are used by central institutions either to compile aggregates and/or they are put together and made available in a uniform manner (e.g. on-line or on a CD-ROM or through file transfers). Therefore, central institutions receive data from other institutions and, usually, they also "disseminate" data to individual and/or institutions for end-use. Within a country, a NSI or a national central bank (NCB) plays, of course, a central institution role as it collects data from other entities and it disseminates statistical information to end users. In SDMX the role of central institution is very important: every statistical message is based on underlying structural definitions (statistical concepts, code lists, DSDs) which have been devised by a particular agency, usually a central institution. Such an institution plays the role of the reference "structural definitions maintenance agency" for the corresponding messages which are exchanged. Of course, two institutions could exchange data using/referring to structural information devised by a third institution.146 +==== 3.4.1.1 Central Institutions and Their Role in Statistical Data Exchanges ==== 157 157 148 +Central institutions are the organisations to which other partner institutions "report" statistics. These statistics are used by central institutions either to compile aggregates and/or they are put together and made available in a uniform manner (e.g. on-line or on a CD-ROM or through file transfers). Therefore, central institutions receive data from other institutions and, usually, they also "disseminate" data to individual and/or institutions for end-use. Within a country, a NSI or a national central bank (NCB) plays, of course, a central institution role as it collects data from other entities and it disseminates statistical information to end users. In SDMX the role of central institution is very important: every statistical message is based on underlying structural definitions (statistical concepts, code lists, DSDs) which have been devised by a particular agency, usually a central institution. Such an institution plays the role of the reference "structural definitions maintenance agency" for the corresponding messages which are exchanged. Of course, two institutions could exchange data using/referring to structural information devised by a third institution. 149 + 158 158 Central institutions can play a double role: 159 159 160 160 * collecting and further disseminating statistics; 161 161 * devising structural definitions for use in data exchanges. 162 162 163 - **3.4.1.2 Defining Data Structure Definitions (DSDs)**155 +==== 3.4.1.2 Defining Data Structure Definitions (DSDs) ==== 164 164 165 165 The following guidelines are suggested for building a DSD. However, it is expected that these guidelines will be considered by central institutions when devising new DSDs. 166 166 167 -=== Dimensions, Attributes and Code Lists === 159 +(% class="wikigeneratedid" id="HDimensions2CAttributesandCodeLists" %) 160 +__Dimensions, Attributes and Code Lists__ 168 168 169 169 **//Avoid dimensions that are not appropriate for all the series in the data structure definition.//** If some dimensions are not applicable (this is evident from the need to have a code in a code list which is marked as “not applicable”, “not relevant” or “total”) for some series then consider moving these series to a new data structure definition in which these dimensions are dropped from the key structure. This is a judgement call as it is sometimes difficult to achieve this without increasing considerably the number of DSDs. 170 170 ... ... @@ -194,7 +194,8 @@ 194 194 195 195 The same code list can be used for several statistical concepts, within a data structure definition or across DSDs. Note that SDMX has recognised that these classifications are often quite large and the usage of codes in any one DSD is only a small extract of the full code list. In this version of the standard it is possible to exchange and disseminate a **partial code list** which is extracted from the full code list and which supports the dimension values valid for a particular DSD. 196 196 197 -=== Data Structure Definition Structure === 190 +(% class="wikigeneratedid" id="HDataStructureDefinitionStructure" %) 191 +__Data Structure Definition Structure__ 198 198 199 199 The following items have to be specified by a structural definitions maintenance agency when defining a new data structure definition: 200 200 ... ... @@ -224,7 +224,7 @@ 224 224 * code list name 225 225 * code values and descriptions 226 226 227 -Definition of data flow definitions. 221 +Definition of data flow definitions. Two (or more) partners performing data exchanges in a certain context need to agree on: 228 228 229 229 * the list of data set identifiers they will be using; 230 230 * for each data flow: ... ... @@ -231,10 +231,12 @@ 231 231 * its content and description 232 232 * the relevant DSD that defines the structure of the data reported or disseminated according the the dataflow definition 233 233 234 - **3.4.1.3 Exchanging Attributes**228 +==== 3.4.1.3 Exchanging Attributes ==== 235 235 236 - **//3.4.1.3.1 Attributes on series, sibling and data set level //**//Staticproperties//.230 +===== //3.4.1.3.1 Attributes on series, sibling and data set level // ===== 237 237 232 +//Static properties//. 233 + 238 238 * Upon creation of a series the sender has to provide to the receiver values for all mandatory attributes. In case they are available, values for conditional attributes should also be provided. Whereas initially this information may be provided by means other than SDMX-ML or SDMX-EDI messages (e.g. paper, telephone) it is expected that partner institutions will be in a position to provide this information in SDMX-ML or SDMX-EDI format over time. 239 239 * A centre may agree with its data exchange partners special procedures for authorising the setting of attributes' initial values. 240 240 * Attribute values at a data set level are set and maintained exclusively by the centre administrating the exchanged data set. ... ... @@ -251,21 +251,21 @@ 251 251 * If the “observation status” changes and the observation remains unchanged, both components would have to be reported. 252 252 * For Data Structure Definitions having also the observation level attributes “observation confidentiality” and "observation pre-break" defined, this rule applies to these attribute as well: if an institution receives from another institution an observation with an observation status attribute only attached, this means that the associated observation confidentiality and prebreak observation attributes either never existed or from now they do not have a value for this observation. 253 253 254 -=== =3.4.2=250 +=== 3.4.2 Best Practices for Batch Data Exchange === 255 255 256 - **3.4.2.1 Introduction**252 +==== 3.4.2.1 Introduction ==== 257 257 258 258 Batch data exchange is the exchange and maintenance of entire databases between counterparties. It is an activity that often employs SDMX-EDI formats, and might also use the SDMX-ML DSD-specific data set. The following points apply equally to both formats. 259 259 260 - **3.4.2.2 Positioning of the Dimension "Frequency"**256 +==== 3.4.2.2 Positioning of the Dimension "Frequency" ==== 261 261 262 262 The position of the “frequency” dimension is unambiguously identified in the data structure definition. Moreover, most central institutions devising structural definitions have decided to assign to this dimension the first position in the key structure. This facilitates the easy identification of this dimension, something that it is necessary to frequency's crucial role in several database systems and in attaching attributes at the “sibling” group level. 263 263 264 - **3.4.2.3 Identification of Data Structure Definitions (DSDs)**260 +==== 3.4.2.3 Identification of Data Structure Definitions (DSDs) ==== 265 265 266 266 In order to facilitate the easy and immediate recognition of the structural definition maintenance agency that defined a data structure definition, most central institutions devising structural definitions use the first characters of the data structure definition identifiers to identify their institution: e.g. BIS_EER, EUROSTAT_BOP_01, ECB_BOP1, etc. 267 267 268 - **3.4.2.4 Identification of the Data Flows**264 +==== 3.4.2.4 Identification of the Data Flows ==== 269 269 270 270 In order to facilitate the easy and immediate recognition of the institution administrating a data flow definitions, many central institutions prefer to use the first characters of the data flow definition identifiers to identify their institution: e.g. BIS_EER, ECB_BOP1, ECB_BOP1, etc. Note that in GESMES/TS the Data Set plays the role of the data flow definition (see //DataSet //in the SDMX-IM//)//. 271 271 ... ... @@ -273,7 +273,7 @@ 273 273 274 274 Note that the role of the Data Flow (called //DataflowDefintion// in the model) and Data Set is very specific in the model, and the terminology used may not be the same as used in all organisations, and specifically the term Data Set is used differently in SDMX than in GESMES/TS. Essentially the GESMES/TS term "Data Set" is, in SDMX, the "Dataflow Definition" whist the term "Data Set" in SDMX is used to describe the "container" for an instance of the data. 275 275 276 - **3.4.2.5 Special Issues**272 +==== 3.4.2.5 Special Issues ==== 277 277 278 278 ===== 3.4.2.5.1 "Frequency" related issues ===== 279 279 ... ... @@ -284,7 +284,6 @@ 284 284 285 285 **//Tick data.//** The issue of data collected at irregular intervals at a higher than daily frequency (e.g. tick-by-tick data) is not discussed here either. However, for data exchange purposes, such series can already be exchanged in the SDMX-EDI format by using the option to send observations with the associated time stamp. 286 286 287 - 288 288 = 4 General Notes for Implementers = 289 289 290 290 This section discusses a number of topics other than the exchange of data sets in SDMX-ML and SDMX-EDI. Supported only in SDMX-ML, these topics include the use of the reference metadata mechanism in SDMX, the use of Structure Sets and Reporting Taxonomies, the use of Processes, a discussion of time and data-typing, and some of the conventional mechanisms within the SDMX-ML Structure message regarding versioning and external referencing. ... ... @@ -295,39 +295,31 @@ 295 295 296 296 There are several different representations in SDMX-ML, taken from XML Schemas and common programming languages. The table below describes the various representations which are found in SDMX-ML, and their equivalents. 297 297 298 -|**SDMX-ML Data Type**|**XML Schema Data Type**|**.NET Framework Type**|((( 299 -**Java Data Type** 300 - 301 -**~ ** 293 +(% style="width:912.294px" %) 294 +|(% style="width:172px" %)**SDMX-ML Data Type**|(% style="width:204px" %)**XML Schema Data Type**|(% style="width:189px" %)**.NET Framework Type**|(% style="width:342px" %)((( 295 +**Java Data Type ** 302 302 ))) 303 -|String|xsd:string|System.String|java.lang.String 304 -|Big Integer|xsd:integer|System.Decimal|java.math.BigInteg er 305 -|Integer|xsd:int|System.Int32|int 306 -|Long|xsd.long|System.Int64|long 307 -|Short|xsd:short|System.Int16|short 308 -|Decimal|xsd:decimal|System.Decimal|java.math.BigDecim al 309 -|Float|xsd:float|System.Single|float 310 -|Double|xsd:double|System.Double|double 311 -|Boolean|xsd:boolean|System.Boolean|boolean 312 -|URI|xsd:anyURI|System.Uri|Java.net.URI or java.lang.String 313 -|DateTime|xsd:dateTime|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 314 -|Time|xsd:time|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 315 -|GregorianYear|xsd:gYear|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 316 -|GregorianMont h|xsd:gYearMont h|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 317 -|GregorianDay|xsd:date|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 318 -|((( 319 -Day, 297 +|(% style="width:172px" %)String|(% style="width:204px" %)xsd:string|(% style="width:189px" %)System.String|(% style="width:342px" %)java.lang.String 298 +|(% style="width:172px" %)Big Integer|(% style="width:204px" %)xsd:integer|(% style="width:189px" %)System.Decimal|(% style="width:342px" %)java.math.BigInteg er 299 +|(% style="width:172px" %)Integer|(% style="width:204px" %)xsd:int|(% style="width:189px" %)System.Int32|(% style="width:342px" %)int 300 +|(% style="width:172px" %)Long|(% style="width:204px" %)xsd.long|(% style="width:189px" %)System.Int64|(% style="width:342px" %)long 301 +|(% style="width:172px" %)Short|(% style="width:204px" %)xsd:short|(% style="width:189px" %)System.Int16|(% style="width:342px" %)short 302 +|(% style="width:172px" %)Decimal|(% style="width:204px" %)xsd:decimal|(% style="width:189px" %)System.Decimal|(% style="width:342px" %)java.math.BigDecim al 303 +|(% style="width:172px" %)Float|(% style="width:204px" %)xsd:float|(% style="width:189px" %)System.Single|(% style="width:342px" %)float 304 +|(% style="width:172px" %)Double|(% style="width:204px" %)xsd:double|(% style="width:189px" %)System.Double|(% style="width:342px" %)double 305 +|(% style="width:172px" %)Boolean|(% style="width:204px" %)xsd:boolean|(% style="width:189px" %)System.Boolean|(% style="width:342px" %)boolean 306 +|(% style="width:172px" %)URI|(% style="width:204px" %)xsd:anyURI|(% style="width:189px" %)System.Uri|(% style="width:342px" %)Java.net.URI or java.lang.String 307 +|(% style="width:172px" %)DateTime|(% style="width:204px" %)xsd:dateTime|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 308 +|(% style="width:172px" %)Time|(% style="width:204px" %)xsd:time|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 309 +|(% style="width:172px" %)GregorianYear|(% style="width:204px" %)xsd:gYear|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 310 +|(% style="width:172px" %)GregorianMonth|(% style="width:204px" %)xsd:gYearMonth|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 311 +|(% style="width:172px" %)GregorianDay|(% style="width:204px" %)xsd:date|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 312 +|(% style="width:172px" %)((( 313 +Day, MonthDay, Month 314 +)))|(% style="width:204px" %)xsd:g*|(% style="width:189px" %)System.DateTime|(% style="width:342px" %)javax.xml.datatype .XMLGregorianCalen dar 315 +|(% style="width:172px" %)Duration|(% style="width:204px" %)xsd:duration |(% style="width:189px" %)System.TimeSpa|(% style="width:342px" %)javax.xml.datatype 316 +|(% style="width:172px" %) |(% style="width:204px" %) |(% style="width:189px" %)n|(% style="width:342px" %).Duration 320 320 321 -MonthDay, Month 322 -)))|xsd:g*|System.DateTim e|javax.xml.datatype .XMLGregorianCalen dar 323 -|Duration|xsd:duration |System.TimeSpa|javax.xml.datatype 324 -|**SDMX-ML Data Type**|**XML Schema Data Type**|**.NET Framework Type**|((( 325 -**Java Data Type** 326 - 327 -**~ ** 328 -))) 329 -| | |n|.Duration 330 - 331 331 There are also a number of SDMX-ML data types which do not have these direct correspondences, often because they are composite representations or restrictions of a broader data type. For most of these, there are simple types which can be referenced from the SDMX schemas, for others a derived simple type will be necessary: 332 332 333 333 * AlphaNumeric (common:AlphaNumericType, string which only allows A-z and 0-9) ... ... @@ -353,10 +353,8 @@ 353 353 * KeyValues (common:DataKeyType) 354 354 * IdentifiableReference (types for each identifiable object) 355 355 * DataSetReference (common:DataSetReferenceType) 356 -* AttachmentConstraintReference 343 +* AttachmentConstraintReference (common:AttachmentConstraintReferenceType) 357 357 358 -(common:AttachmentConstraintReferenceType) 359 - 360 360 Data types also have a set of facets: 361 361 362 362 * isSequence = true | false (indicates a sequentially increasing value) ... ... @@ -378,7 +378,7 @@ 378 378 379 379 == 4.2 Time and Time Format == 380 380 381 -=== =4.2.1 Introduction=366 +=== 4.2.1 Introduction === 382 382 383 383 First, it is important to recognize that most observation times are a period. SDMX specifies precisely how Time is handled. 384 384 ... ... @@ -386,50 +386,47 @@ 386 386 387 387 The hierarchy of time formats is as follows (**bold** indicates a category which is made up of multiple formats, //italic// indicates a distinct format): 388 388 389 -* **Observational Time Period **o **Standard Time Period** 374 +* **Observational Time Period** 375 +** **Standard Time Period** 376 +*** **Basic Time Period** 377 +**** **Gregorian Time Period** 378 +**** //Date Time// 379 +*** **Reporting Time Period** 380 +** //Time Range// 390 390 391 - § **Basic Time Period** 392 - 393 -* **Gregorian Time Period** 394 -* //Date Time// 395 - 396 -§ **Reporting Time Period **o //Time Range// 397 - 398 398 The details of these time period categories and of the distinct formats which make them up are detailed in the sections to follow. 399 399 400 -=== =4.2.2 Observational Time Period ====384 +=== 4.2.2 Observational Time Period === 401 401 402 402 This is the superset of all time representations in SDMX. This allows for time to be expressed as any of the allowable formats. 403 403 404 -=== =4.2.3 Standard Time Period ====388 +=== 4.2.3 Standard Time Period === 405 405 406 406 This is the superset of any predefined time period or a distinct point in time. A time period consists of a distinct start and end point. If the start and end of a period are expressed as date instead of a complete date time, then it is implied that the start of the period is the beginning of the start day (i.e. 00:00:00) and the end of the period is the end of the end day (i.e. 23:59:59). 407 407 408 -=== =4.2.4 Gregorian Time Period ====392 +=== 4.2.4 Gregorian Time Period === 409 409 410 410 A Gregorian time period is always represented by a Gregorian year, year-month, or day. These are all based on ISO 8601 dates. The representation in SDMX-ML messages and the period covered by each of the Gregorian time periods are as follows: 411 411 412 -**Gregorian Year:** 413 - 396 +**Gregorian Year:** 414 414 Representation: xs:gYear (YYYY) 398 +Period: the start of January 1 to the end of December 31 415 415 416 -Period: the start of January 1 to the end of December 31 **Gregorian Year Month**: 417 - 400 +**Gregorian Year Month**: 418 418 Representation: xs:gYearMonth (YYYY-MM) 402 +Period: the start of the first day of the month to end of the last day of the month 419 419 420 -Period: the start of the first day of the month to end of the last day of the month **Gregorian Day**: 421 - 404 +**Gregorian Day**: 422 422 Representation: xs:date (YYYY-MM-DD) 423 - 424 424 Period: the start of the day (00:00:00) to the end of the day (23:59:59) 425 425 426 -=== =4.2.5 Date Time ====408 +=== 4.2.5 Date Time === 427 427 428 428 This is used to unambiguously state that a date-time represents an observation at a single point in time. Therefore, if one wants to use SDMX for data which is measured at a distinct point in time rather than being reported over a period, the date-time representation can be used. 429 429 430 -Representation: xs:dateTime (YYYY-MM-DDThh:mm:ss)[[^^~[1~]^^>>path:#_ftn1]] 412 +Representation: xs:dateTime (YYYY-MM-DDThh:mm:ss)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[1~]^^>>path:#_ftn1]] 431 431 432 -=== =4.2.6 Standard Reporting Period ====414 +=== 4.2.6 Standard Reporting Period === 433 433 434 434 Standard reporting periods are periods of time in relation to a reporting year. Each of these standard reporting periods has a duration (based on the ISO 8601 definition) associated with it. The general format of a reporting period is as follows: 435 435 ... ... @@ -436,75 +436,52 @@ 436 436 [REPORTING_YEAR]-[PERIOD_INDICATOR][PERIOD_VALUE] 437 437 438 438 Where: 439 - 440 440 REPORTING_YEAR represents the reporting year as four digits (YYYY) PERIOD_INDICATOR identifies the type of period which determines the duration of the period 441 - 442 442 PERIOD_VALUE indicates the actual period within the year 443 443 444 444 The following section details each of the standard reporting periods defined in SDMX: 445 445 446 -**Reporting Year**: 447 - 448 - Period Indicator: A 449 - 426 +**Reporting Year**: 427 +Period Indicator: A 450 450 Period Duration: P1Y (one year) 451 - 452 452 Limit per year: 1 430 +Representation: common:ReportingYearType (YYYY-A1, e.g. 2000-A1) 453 453 454 -Representation: common:ReportingYearType (YYYY-A1, e.g. 2000-A1) **Reporting Semester:** 455 - 456 - Period Indicator: S 457 - 432 +**Reporting Semester:** 433 +Period Indicator: S 458 458 Period Duration: P6M (six months) 459 - 460 460 Limit per year: 2 436 +Representation: common:ReportingSemesterType (YYYY-Ss, e.g. 2000-S2) 461 461 462 -Representation: common:ReportingSemesterType (YYYY-Ss, e.g. 2000-S2) **Reporting Trimester:** 463 - 464 - Period Indicator: T 465 - 438 +**Reporting Trimester:** 439 +Period Indicator: T 466 466 Period Duration: P4M (four months) 467 - 468 468 Limit per year: 3 442 +Representation: common:ReportingTrimesterType (YYYY-Tt, e.g. 2000-T3) 469 469 470 -Representation: common:ReportingTrimesterType (YYYY-Tt, e.g. 2000-T3) **Reporting Quarter:** 471 - 472 - Period Indicator: Q 473 - 444 +**Reporting Quarter:** 445 +Period Indicator: Q 474 474 Period Duration: P3M (three months) 475 - 476 476 Limit per year: 4 448 +Representation: common:ReportingQuarterType (YYYY-Qq, e.g. 2000-Q4) 477 477 478 -Representation: common:ReportingQuarterType (YYYY-Qq, e.g. 2000-Q4) **Reporting Month**: 479 - 450 +**Reporting Month**: 480 480 Period Indicator: M 481 - 482 482 Period Duration: P1M (one month) 483 - 484 484 Limit per year: 1 485 - 486 486 Representation: common:ReportingMonthType (YYYY-Mmm, e.g. 2000-M12) Notes: The reporting month is always represented as two digits, therefore 1-9 are 0 padded (e.g. 01). This allows the values to be sorted chronologically using textual sorting methods. 487 487 488 488 **Reporting Week**: 489 - 490 490 Period Indicator: W 491 - 492 492 Period Duration: P7D (seven days) 493 - 494 494 Limit per year: 53 495 - 496 496 Representation: common:ReportingWeekType (YYYY-Www, e.g. 2000-W53) 461 +Notes: There are either 52 or 53 weeks in a reporting year. This is based on the ISO 8601 definition of a week (Monday - Saturday), where the first week of a reporting year is defined as the week with the first Thursday on or after the reporting year start day.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[2~]^^>>path:#_ftn2]](%%) The reporting week is always represented as two digits, therefore 1-9 are 0 padded (e.g. 01). This allows the values to be sorted chronologically using textual sorting methods. 497 497 498 -Notes: There are either 52 or 53 weeks in a reporting year. This is based on the ISO 8601 definition of a week (Monday - Saturday), where the first week of a reporting year is defined as the week with the first Thursday on or after the reporting year start day.[[^^~[2~]^^>>path:#_ftn2]] The reporting week is always represented as two digits, therefore 1-9 are 0 padded (e.g. 01). This allows the values to be sorted chronologically using textual sorting methods. 499 - 500 500 **Reporting Day**: 501 - 502 502 Period Indicator: D 503 - 504 504 Period Duration: P1D (one day) 505 - 506 506 Limit per year: 366 507 - 508 508 Representation: common:ReportingDayType (YYYY-Dddd, e.g. 2000-D366) Notes: There are either 365 or 366 days in a reporting year, depending on whether the reporting year includes leap day (February 29). The reporting day is always represented as three digits, therefore 1-99 are 0 padded (e.g. 001). 509 509 510 510 This allows the values to be sorted chronologically using textual sorting methods. ... ... @@ -515,143 +515,109 @@ 515 515 516 516 Since the duration and the reporting year start day are known for any reporting period, it is possible to relate any reporting period to a distinct calendar period. The actual Gregorian calendar period covered by the reporting period can be computed as follows (based on the standard format of [REPROTING_YEAR][PERIOD_INDICATOR][PERIOD_VALUE] and the reporting year start day as [REPORTING_YEAR_START_DAY]): 517 517 518 -1. **Determine [REPORTING_YEAR_BASE]:** 519 - 477 +**~1. Determine [REPORTING_YEAR_BASE]:** 520 520 Combine [REPORTING_YEAR] of the reporting period value (YYYY) with [REPORTING_YEAR_START_DAY] (MM-DD) to get a date (YYYY-MM-DD). 521 - 522 522 This is the [REPORTING_YEAR_START_DATE] 523 - 524 -**a) If the [PERIOD_INDICATOR] is W:** 525 - 526 -1. 527 -11. 528 -111. 529 -1111. **If [REPORTING_YEAR_START_DATE] is a Friday, Saturday, or Sunday:** 530 - 480 +**a) If the [PERIOD_INDICATOR] is W: 481 +~1. If [REPORTING_YEAR_START_DATE] is a Friday, Saturday, or Sunday:** 531 531 Add^^3^^ (P3D, P2D, or P1D respectively) to the [REPORTING_YEAR_START_DATE]. The result is the [REPORTING_YEAR_BASE]. 532 532 533 -1. 534 -11. 535 -111. 536 -1111. **If [REPORTING_YEAR_START_DATE] is a Monday, Tuesday, Wednesday, or Thursday:** 537 - 484 +2. **If [REPORTING_YEAR_START_DATE] is a Monday, Tuesday, Wednesday, or Thursday:** 538 538 Add^^3^^ (P0D, -P1D, -P2D, or -P3D respectively) to the [REPORTING_YEAR_START_DATE]. The result is the [REPORTING_YEAR_BASE]. 486 +b) **Else:** 487 +The [REPORTING_YEAR_START_DATE] is the [REPORTING_YEAR_BASE] 539 539 540 - b)**Else:**489 +**2. Determine [PERIOD_DURATION]:** 541 541 542 -The [REPORTING_YEAR_START_DATE] is the [REPORTING_YEAR_BASE]. 491 +a) If the [PERIOD_INDICATOR] is A, the [PERIOD_DURATION] is P1Y. 492 +b) If the [PERIOD_INDICATOR] is S, the [PERIOD_DURATION] is P6M. 493 +c) If the [PERIOD_INDICATOR] is T, the [PERIOD_DURATION] is P4M. 494 +d) If the [PERIOD_INDICATOR] is Q, the [PERIOD_DURATION] is P3M. 495 +e) If the [PERIOD_INDICATOR] is M, the [PERIOD_DURATION] is P1M. 496 +f) If the [PERIOD_INDICATOR] is W, the [PERIOD_DURATION] is P7D. 497 +g) If the [PERIOD_INDICATOR] is D, the [PERIOD_DURATION] is P1D. 543 543 544 -1. **Determine [PERIOD_DURATION]:** 545 -11. 546 -111. If the [PERIOD_INDICATOR] is A, the [PERIOD_DURATION] is P1Y. 547 -111. If the [PERIOD_INDICATOR] is S, the [PERIOD_DURATION] is P6M. 548 -111. If the [PERIOD_INDICATOR] is T, the [PERIOD_DURATION] is P4M. 549 -111. If the [PERIOD_INDICATOR] is Q, the [PERIOD_DURATION] is P3M. 550 -111. If the [PERIOD_INDICATOR] is M, the [PERIOD_DURATION] is P1M. 551 -111. If the [PERIOD_INDICATOR] is W, the [PERIOD_DURATION] is P7D. 552 -111. If the [PERIOD_INDICATOR] is D, the [PERIOD_DURATION] is P1D. 553 -1. **Determine [PERIOD_START]:** 499 +**3. Determine [PERIOD_START]:** 500 +Subtract one from the [PERIOD_VALUE] and multiply this by the [PERIOD_DURATION]. Add[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[3~]^^>>path:#_ftn3]](%%) this to the [REPORTING_YEAR_BASE]. The result is the [PERIOD_START]. 554 554 555 -Subtract one from the [PERIOD_VALUE] and multiply this by the [PERIOD_DURATION]. Add[[^^~[3~]^^>>path:#_ftn3]] this to the [REPORTING_YEAR_BASE]. The result is the [PERIOD_START]. 556 - 557 -1. **Determine the [PERIOD_END]:** 558 - 502 +**4. Determine the [PERIOD_END]:** 559 559 Multiply the [PERIOD_VALUE] by the [PERIOD_DURATION]. Add^^3^^ this to the [REPORTING_YEAR_BASE] add^^3^^ -P1D. The result is the [PERIOD_END]. 560 560 561 561 For all of these ranges, the bounds include the beginning of the [PERIOD_START] (i.e. 00:00:00) and the end of the [PERIOD_END] (i.e. 23:59:59). 562 562 563 -**Examples: 507 +**Examples:** 564 564 565 565 **2010-Q2, REPORTING_YEAR_START_DAY = ~-~-07-01 (July 1)** 566 - 567 567 ~1. [REPORTING_YEAR_START_DATE] = 2010-07-01 568 - 569 569 b) [REPORTING_YEAR_BASE] = 2010-07-01 570 - 571 -1. [PERIOD_DURATION] = P3M 572 -1. (2-1) * P3M = P3M 573 - 512 +[PERIOD_DURATION] = P3M 513 +(2-1) * P3M = P3M 574 574 2010-07-01 + P3M = 2010-10-01 575 - 576 576 [PERIOD_START] = 2010-10-01 577 - 578 578 4. 2 * P3M = P6M 579 - 580 580 2010-07-01 + P6M = 2010-13-01 = 2011-01-01 581 - 582 582 2011-01-01 + -P1D = 2010-12-31 583 - 584 584 [PERIOD_END] = 2011-12-31 585 585 586 586 The actual calendar range covered by 2010-Q2 (assuming the reporting year begins July 1) is 2010-10-01T00:00:00/2010-12-31T23:59:59 587 587 588 588 **2011-W36, REPORTING_YEAR_START_DAY = ~-~-07-01 (July 1)** 589 - 590 590 ~1. [REPORTING_YEAR_START_DATE] = 2010-07-01 591 - 592 592 a) 2011-07-01 = Friday 593 - 594 594 2011-07-01 + P3D = 2011-07-04 595 - 596 596 [REPORTING_YEAR_BASE] = 2011-07-04 597 - 598 -1. [PERIOD_DURATION] = P7D 599 -1. (36-1) * P7D = P245D 600 - 528 +2. [PERIOD_DURATION] = P7D 529 +3. (36-1) * P7D = P245D 601 601 2011-07-04 + P245D = 2012-03-05 602 - 603 603 [PERIOD_START] = 2012-03-05 604 - 605 605 4. 36 * P7D = P252D 606 - 607 607 2011-07-04 + P252D =2012-03-12 608 - 609 609 2012-03-12 + -P1D = 2012-03-11 610 - 611 611 [PERIOD_END] = 2012-03-11 612 612 613 613 The actual calendar range covered by 2011-W36 (assuming the reporting year begins July 1) is 2012-03-05T00:00:00/2012-03-11T23:59:59 614 614 615 -=== =4.2.7 Distinct Range ====539 +=== 4.2.7 Distinct Range === 616 616 617 617 In the case that the reporting period does not fit into one of the prescribe periods above, a distinct time range can be used. The value of these ranges is based on the ISO 8601 time interval format of start/duration. Start can be expressed as either an ISO 8601 date or a date-time, and duration is expressed as an ISO 8601 duration. However, the duration can only be postive. 618 618 619 -=== =4.2.8 Time Format ====543 +=== 4.2.8 Time Format === 620 620 621 621 In version 2.0 of SDMX there is a recommendation to use the time format attribute to gives additional information on the way time is represented in the message. Following an appraisal of its usefulness this is no longer required. However, it is still possible, if required , to include the time format attribute in SDMX-ML. 622 622 623 -|**Code**|**Format** 624 -|**OTP**|Observational Time Period: Superset of all SDMX time formats (Gregorian Time Period, Reporting Time Period, and Time Range) 625 -|**STP**|Standard Time Period: Superset of Gregorian and Reporting Time Periods 626 -|**GTP**|Superset of all Gregorian Time Periods and date-time 627 -|**RTP**|Superset of all Reporting Time Periods 628 -|**TR**|Time Range: Start time and duration (YYYY-MMDD(Thh:mm:ss)?/<duration>) 629 -|**GY**|Gregorian Year (YYYY) 630 -|**GTM**|Gregorian Year Month (YYYY-MM) 631 -|**GD**|Gregorian Day (YYYY-MM-DD) 632 -|**DT**|Distinct Point: date-time (YYYY-MM-DDThh:mm:ss) 633 -|**RY**|Reporting Year (YYYY-A1) 634 -|**RS**|Reporting Semester (YYYY-Ss) 635 -|**RT**|Reporting Trimester (YYYY-Tt) 636 -|**RQ**|Reporting Quarter (YYYY-Qq) 637 -|**RM**|Reporting Month (YYYY-Mmm) 638 -|**Code**|**Format** 639 -|**RW**|Reporting Week (YYYY-Www) 640 -|**RD**|Reporting Day (YYYY-Dddd) 547 +(% style="width:1049.29px" %) 548 +|**Code**|(% style="width:926px" %)**Format** 549 +|**OTP**|(% style="width:926px" %)Observational Time Period: Superset of all SDMX time formats (Gregorian Time Period, Reporting Time Period, and Time Range) 550 +|**STP**|(% style="width:926px" %)Standard Time Period: Superset of Gregorian and Reporting Time Periods 551 +|**GTP**|(% style="width:926px" %)Superset of all Gregorian Time Periods and date-time 552 +|**RTP**|(% style="width:926px" %)Superset of all Reporting Time Periods 553 +|**TR**|(% style="width:926px" %)Time Range: Start time and duration (YYYY-MMDD(Thh:mm:ss)?/<duration>) 554 +|**GY**|(% style="width:926px" %)Gregorian Year (YYYY) 555 +|**GTM**|(% style="width:926px" %)Gregorian Year Month (YYYY-MM) 556 +|**GD**|(% style="width:926px" %)Gregorian Day (YYYY-MM-DD) 557 +|**DT**|(% style="width:926px" %)Distinct Point: date-time (YYYY-MM-DDThh:mm:ss) 558 +|**RY**|(% style="width:926px" %)Reporting Year (YYYY-A1) 559 +|**RS**|(% style="width:926px" %)Reporting Semester (YYYY-Ss) 560 +|**RT**|(% style="width:926px" %)Reporting Trimester (YYYY-Tt) 561 +|**RQ**|(% style="width:926px" %)Reporting Quarter (YYYY-Qq) 562 +|**RM**|(% style="width:926px" %)Reporting Month (YYYY-Mmm) 563 +|**Code**|(% style="width:926px" %)**Format** 564 +|**RW**|(% style="width:926px" %)Reporting Week (YYYY-Www) 565 +|**RD**|(% style="width:926px" %)Reporting Day (YYYY-Dddd) 641 641 642 - 567 +**Table 1: SDMX-ML Time Format Codes** 643 643 644 -=== =4.2.9 Transformation between SDMX-ML and SDMX-EDI ====569 +=== 4.2.9 Transformation between SDMX-ML and SDMX-EDI === 645 645 646 646 When converting SDMX-ML data structure definitions to SDMX-EDI data structure definitions, only the identifier of the time format attribute will be retained. The representation of the attribute will be converted from the SDMX-ML format to the fixed SDMX-EDI code list. If the SDMX-ML data structure definition does not define a time format attribute, then one will be automatically created with the identifier "TIME_FORMAT". 647 647 648 -When converting SDMX-ML data to SDMX-EDI, the source time format attribute will be irrelevant. Since the SDMX-ML time representation types are not ambiguous, the target time format can be determined from the source time value directly. For example, if the SDMX-ML time is 2000-Q2 the SDMX-EDI format will always be 608/708 (depending on whether the target series contains one observation or a range of observations) 573 +When converting SDMX-ML data to SDMX-EDI, the source time format attribute will be irrelevant. Since the SDMX-ML time representation types are not ambiguous, the target time format can be determined from the source time value directly. For example, if the SDMX-ML time is 2000-Q2 the SDMX-EDI format will always be 608/708 (depending on whether the target series contains one observation or a range of observations). 649 649 650 650 When converting a data structure definition originating in SDMX-EDI, the time format attribute should be ignored, as it serves no purpose in SDMX-ML. 651 651 652 652 When converting data from SDMX-EDI to SDMX-ML, the source time format is only necessary to determine the format of the target time value. For example, a source time format of will result in a target time in the format YYYY-Ss whereas a source format of will result in a target time value in the format YYYY-Qq. 653 653 654 -=== ==579 +=== 4.2.10 Time Zones === 655 655 656 656 In alignment with ISO 8601, SDMX allows the specification of a time zone on all time periods and on the reporting year start day. If a time zone is provided on a reporting year start day, then the same time zone (or none) should be reported for each reporting time period. If the reporting year start day and the reporting period time zone differ, the time zone of the reporting period will take precedence. Examples of each format with time zones are as follows (time zone indicated in bold): 657 657 ... ... @@ -672,7 +672,7 @@ 672 672 673 673 According to ISO 8601, a date without a time-zone is considered "local time". SDMX assumes that local time is that of the sender of the message. In this version of SDMX, an optional field is added to the sender definition in the header for specifying a time zone. This field has a default value of 'Z' (UTC). This determination of local time applies for all dates in a message. 674 674 675 -=== ==600 +=== 4.2.11 Representing Time Spans Elsewhere === 676 676 677 677 It has been possible since SDMX 2.0 for a Component to specify a representation of a time span. Depending on the format of the data message, this resulted in either an element with 2 XML attributes for holding the start time and the duration or two separate XML attributes based on the underlying Component identifier. For example if REF_PERIOD were given a representation of time span, then in the Compact data format, it would be represented by two XML attributes; REF_PERIODStartTime (holding the start) and REF_PERIOD (holding the duration). If a new simple type is introduced in the SDMX schemas that can hold ISO 8601 time intervals, then this will no longer be necessary. What was represented as this: 678 678 ... ... @@ -682,30 +682,29 @@ 682 682 683 683 <Series REF_PERIOD="2000-01-01T00:00:00/P2M"/> 684 684 685 -=== ==610 +=== 4.2.12 Notes on Formats === 686 686 687 687 There is no ambiguity in these formats so that for any given value of time, the category of the period (and thus the intended time period range) is always clear. It should also be noted that by utilizing the ISO 8601 format, and a format loosely based on it for the report periods, the values of time can easily be sorted chronologically without additional parsing. 688 688 689 -=== ==614 +=== 4.2.13 Effect on Time Ranges === 690 690 691 691 All SDMX-ML data messages are capable of functioning in a manner similar to SDMX-EDI if the Dimension at the observation level is time: the time period for the first observation can be stated and the rest of the observations can omit the time value as it can be derived from the start time and the frequency. Since the frequency can be determined based on the actual format of the time value for everything but distinct points in time and time ranges, this makes is even simpler to process as the interval between time ranges is known directly from the time value. 692 692 693 -=== =4.2.14 Time in Query Messages ====618 +=== 4.2.14 Time in Query Messages === 694 694 695 695 When querying for time values, the value of a time parameter can be provided as any of the Observational Time Period formats and must be paired with an operator. In addition, an explicit value for the reporting year start day can be provided, or this can be set to "Any". This section will detail how systems processing query messages should interpret these parameters. 696 696 697 697 Fundamental to processing a time value parameter in a query message is understanding that all time periods should be handled as a distinct range of time. Since the time parameter in the query is paired with an operator, this is also effectively represents a distinct range of time. Therefore, a system processing the query must simply match the data where the time period for requested parameter is encompassed by the time period resulting from value of the query parameter. The following table details how the operators should be interpreted for any time period provided as a parameter. 698 698 699 -|**Operator**|**Rule** 700 -|Greater Than|Any data after the last moment of the period 701 -|Less Than|Any data before the first moment of the period 702 -|Greater Than or Equal To|((( 703 -Any data on or after the first moment of 704 - 705 -the period 624 +(% style="width:1024.29px" %) 625 +|(% style="width:238px" %)**Operator**|(% style="width:782px" %)**Rule** 626 +|(% style="width:238px" %)Greater Than|(% style="width:782px" %)Any data after the last moment of the period 627 +|(% style="width:238px" %)Less Than|(% style="width:782px" %)Any data before the first moment of the period 628 +|(% style="width:238px" %)Greater Than or Equal To|(% style="width:782px" %)((( 629 +Any data on or after the first moment of the period 706 706 ))) 707 -|Less Than or Equal To|Any data on or before the last moment of the period 708 -|Equal To|Any data which falls on or after the first moment of the period and before or on the last moment of the period 631 +|(% style="width:238px" %)Less Than or Equal To|(% style="width:782px" %)Any data on or before the last moment of the period 632 +|(% style="width:238px" %)Equal To|(% style="width:782px" %)Any data which falls on or after the first moment of the period and before or on the last moment of the period 709 709 710 710 Reporting Time Periods as query parameters are handled based on whether the value of the reportingYearStartDay XML attribute is an explicit month and day or "Any": 711 711 ... ... @@ -765,7 +765,7 @@ 765 765 766 766 2010-D185 or later (reporting year start day ~-~-07-01) 767 767 768 -== 4.3 Structural Metadata Querying Best Practices 692 +== 4.3 Structural Metadata Querying Best Practices == 769 769 770 770 When querying for structural metadata, the ability to state how references should be resolved is quite powerful. However, this mechanism is not always necessary and can create an undue burden on the systems processing the queries if it is not used properly. 771 771 ... ... @@ -773,7 +773,7 @@ 773 773 774 774 When the referenced object is not known, then the reference resolution mechanism could be used. For example, suppose one wanted to find all category schemes and the related categorisations for a given maintenance agency. In this case, one could query for the category scheme by the maintenance agency and specify that parent and sibling references should be resolved. This would result in the categorisations which reference the categories in the matched schemes to be returned, as well as the object which they categorise. 775 775 776 -== 4.4 Versioning and External Referencing 700 +== 4.4 Versioning and External Referencing == 777 777 778 778 Within the SDMX-ML Structure Message, there is a pattern for versioning and external referencing which should be pointed out. The identifiers are qualified by their version numbers – that is, an object with an Agency of “A”, and ID of “X” and a version of “1.0” is a different object than one with an Agency of “A’, an ID of “X”, and a version of “1.1”. 779 779 ... ... @@ -1188,12 +1188,12 @@ 1188 1188 1. Restricts the code list for the CAS Dimension to codes TOT and NAP. 1189 1189 1. Inherits the AGE constraint applied at the level of the DSD. 1190 1190 1191 -=== Provision Agreements CENSUS_CUBE1_IT 1115 +=== Provision Agreements CENSUS_CUBE1_IT === 1192 1192 1193 1193 1. Restricts the codes for the GEO Dimension to IT and its children. 1194 1194 1. Inherits the constraints from Dataflow CENSUS_CUBE1 for the AGE and CAS Dimensions. 1195 1195 1196 -=== Provision Agreements CENSUS_CUBE2_IT 1120 +=== Provision Agreements CENSUS_CUBE2_IT === 1197 1197 1198 1198 1. Restricts the codes for the GEO Dimension to IT and its children. 1199 1199 1. Inherits the constraints from Dataflow CENSUS_CUBE2 for the CAS Dimension. ... ... @@ -1223,7 +1223,7 @@ 1223 1223 1224 1224 == 9.2 Groups and Dimension Groups == 1225 1225 1226 -=== 1150 +=== 9.2.1 Issue === 1227 1227 1228 1228 Version 2.1 introduces a more granular mechanism for specifying the relationship between a Data Attribute and the Dimensions to which the attribute applies. The technical construct for this is the Dimension Group. This Dimension Group has no direct equivalent in versions 2.0 and 1.0 and so the application transforming data from a version 2.1 data set to a version 2.0 or version 1.0 data set must decide to which construct the attribute value, whose Attribute is declared in a Dimension Group, should be attached. The closest construct is the “Series” attachment level and in many cases this is the correct construct to use. 1229 1229 ... ... @@ -1248,7 +1248,7 @@ 1248 1248 1249 1249 == 10.1 Introduction == 1250 1250 1251 -The Validation and Transformation Language (VTL) supports the definition of Transformations, which are algorithms to calculate new data starting from already existing ones[[^^~[4~]^^>>path:#_ftn4]]. The purpose of the VTL in the SDMX context is to enable the: 1175 +The Validation and Transformation Language (VTL) supports the definition of Transformations, which are algorithms to calculate new data starting from already existing ones[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[4~]^^>>path:#_ftn4]](%%). The purpose of the VTL in the SDMX context is to enable the: 1252 1252 1253 1253 * definition of validation and transformation algorithms, in order to specify how to calculate new data from existing ones; 1254 1254 * exchange of the definition of VTL algorithms, also together the definition of the data structures of the involved data (for example, exchange the data structures of a reporting framework together with the validation rules to be applied, exchange the input and output data structures of a calculation task together with the VTL Transformations describing the calculation algorithms); ... ... @@ -1262,7 +1262,7 @@ 1262 1262 1263 1263 This section does not explain the VTL language or any of the content published in the VTL guides. Rather, this is a description of how the VTL can be used in the SDMX context and applied to SDMX artefacts. 1264 1264 1265 -== 10.2 References to SDMX artefacts from VTL statements 1189 +== 10.2 References to SDMX artefacts from VTL statements == 1266 1266 1267 1267 === 10.2.1 Introduction === 1268 1268 ... ... @@ -1272,7 +1272,7 @@ 1272 1272 1273 1273 In any case, the aliases used in the VTL transformations have to be mapped to the 1274 1274 1275 -SDMX artefacts through the VtlMappingScheme and VtlMapping classes (see the section of the SDMX IM relevant to the VTL). A VtlMapping allows specifying the aliases to be used in the VTL transformations, rulesets[[^^~[5~]^^>>path:#_ftn5]] or user defined operators[[^^~[6~]^^>>path:#_ftn6]] to reference SDMX artefacts. A VtlMappingScheme is a container for zero or more VtlMapping. 1199 +SDMX artefacts through the VtlMappingScheme and VtlMapping classes (see the section of the SDMX IM relevant to the VTL). A VtlMapping allows specifying the aliases to be used in the VTL transformations, rulesets[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[5~]^^>>path:#_ftn5]](%%) or user defined operators[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[6~]^^>>path:#_ftn6]](%%) to reference SDMX artefacts. A VtlMappingScheme is a container for zero or more VtlMapping. 1276 1276 1277 1277 The correspondence between an alias and a SDMX artefact must be one-to-one, meaning that a generic alias identifies one and just one SDMX artefact while a SDMX artefact is identified by one and just one alias. In other words, within a VtlMappingScheme an artefact can have just one alias and different artefacts cannot have the same alias. 1278 1278 ... ... @@ -1282,15 +1282,15 @@ 1282 1282 1283 1283 This approach has the advantage that in the VTL code the URN of the referenced artefacts is directly intelligible by a human reader but has the drawback that the references are verbose. 1284 1284 1285 -The SDMX URN[[^^~[7~]^^>>path:#_ftn7]] is the concatenation of the following parts, separated by special symbols like dot, equal, asterisk, comma, and parenthesis:^^ ^^ 1209 +The SDMX URN[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[7~]^^>>path:#_ftn7]](%%) is the concatenation of the following parts, separated by special symbols like dot, equal, asterisk, comma, and parenthesis:^^ ^^ 1286 1286 1287 -* SDMXprefix 1288 -* SDMX-IM-package-name 1289 -* class-name 1290 -* agency-id 1211 +* SDMXprefix 1212 +* SDMX-IM-package-name 1213 +* class-name 1214 +* agency-id 1291 1291 * maintainedobject-id 1292 1292 * maintainedobject-version 1293 -* container-object-id [[^^~[8~]^^>>path:#_ftn8]] 1217 +* container-object-id [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[8~]^^>>path:#_ftn8]] 1294 1294 * object-id 1295 1295 1296 1296 The generic structure of the URN is the following: ... ... @@ -1309,7 +1309,7 @@ 1309 1309 1310 1310 The **agency-id** is the acronym of the agency that owns the definition of the artefact, for example for the Eurostat artefacts the agency-id is “ESTAT”). The agency-id can be composite (for example AgencyA.Dept1.Unit2). 1311 1311 1312 -The **maintainedobject-id** is the name of the maintained object which the artefact belongs to, and in case the artefact itself is maintainable[[^^~[9~]^^>>path:#_ftn9]], coincides with the name of the artefact. Therefore the maintainedobject-id depends on the class of the artefact: 1236 +The **maintainedobject-id** is the name of the maintained object which the artefact belongs to, and in case the artefact itself is maintainable[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[9~]^^>>path:#_ftn9]](%%), coincides with the name of the artefact. Therefore the maintainedobject-id depends on the class of the artefact: 1313 1313 1314 1314 * if the artefact is a ,,Dataflow,,, which is a maintainable class, the maintainedobject-id is the Dataflow name (dataflow-id); 1315 1315 * if the artefact is a Dimension, MeasureDimension, TimeDimension, PrimaryMeasure or DataAttribute, which are not maintainable and belong to the ,,DataStructure,, maintainable class, the maintainedobject-id is the name of the DataStructure (dataStructure-id) which the artefact belongs to; ... ... @@ -1329,7 +1329,7 @@ 1329 1329 1330 1330 * if the artefact is a ,,Concept ,,(the object-id is the name of the ,,Concept,,) 1331 1331 1332 -For example, by using the URN, the VTL transformation that sums two SDMX dataflows DF1 and DF2 and assigns the result to a third persistent dataflow DFR, assuming that DF1, DF2 and DFR are the maintainedobject-id of the three dataflows, that their version is 1.0 and their Agency is AG, would be written as[[^^~[10~]^^>>path:#_ftn10]]: 1256 +For example, by using the URN, the VTL transformation that sums two SDMX dataflows DF1 and DF2 and assigns the result to a third persistent dataflow DFR, assuming that DF1, DF2 and DFR are the maintainedobject-id of the three dataflows, that their version is 1.0 and their Agency is AG, would be written as[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[10~]^^>>path:#_ftn10]](%%): 1333 1333 1334 1334 ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DFR(1.0)’ <- 1335 1335 ... ... @@ -1345,10 +1345,10 @@ 1345 1345 1346 1346 * The **SDMXPrefix** can be omitted for all the SDMX objects, because it is a prefixed string (urn:sdmx:org), always the same for SDMX objects. 1347 1347 * The **SDMX-IM-package-name **can be omitted as well because it can be deduced from the class-name that follows it (the table of the SDMX-IM packages and classes that allows this deduction is in the SDMX 2.1 Standards - Section 5 - Registry Specifications, paragraph 6.2.3). In particular, considering the object classes of the artefacts that VTL can reference, the package is: 1348 -** “datastructure” for the classes Dataflow, Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, 1272 +** “datastructure” for the classes Dataflow, Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, 1349 1349 ** “conceptscheme” for the classes Concept and ConceptScheme o “codelist” for the class Codelist. 1350 -* The **class-name** can be omitted as it can be deduced from the VTL invocation. In particular, starting from the VTL class of the invoked artefact (e.g. dataset, component, identifier, measure, attribute, variable, valuedomain), which is known given the syntax of the invoking VTL operator[[^^~[11~]^^>>path:#_ftn11]], the SDMX class can be deduced from the mapping rules between VTL and SDMX (see the section “Mapping between VTL and SDMX” hereinafter)[[^^~[12~]^^>>path:#_ftn12]]. 1351 -* If the **agency-id** is not specified, it is assumed by default equal to the agency-id of the TransformationScheme, UserDefinedOperatorScheme or RulesetScheme from which the artefact is invoked. For example, the agency-id can be omitted if it is the same as the invoking T,,ransformationScheme,, and cannot be omitted if the artefact comes from another agency.[[^^~[13~]^^>>path:#_ftn13]] Take also into account that, according to the VTL consistency rules, the agency of the result of a ,,Transformation,, must be the same as its ,,TransformationScheme,,, therefore the agency-id can be omitted for all the results (left part of ,,Transformation,, statements). 1274 +* The **class-name** can be omitted as it can be deduced from the VTL invocation. In particular, starting from the VTL class of the invoked artefact (e.g. dataset, component, identifier, measure, attribute, variable, valuedomain), which is known given the syntax of the invoking VTL operator[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[11~]^^>>path:#_ftn11]](%%), the SDMX class can be deduced from the mapping rules between VTL and SDMX (see the section “Mapping between VTL and SDMX” hereinafter)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[12~]^^>>path:#_ftn12]](%%). 1275 +* If the **agency-id** is not specified, it is assumed by default equal to the agency-id of the TransformationScheme, UserDefinedOperatorScheme or RulesetScheme from which the artefact is invoked. For example, the agency-id can be omitted if it is the same as the invoking T,,ransformationScheme,, and cannot be omitted if the artefact comes from another agency.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[13~]^^>>path:#_ftn13]](%%) Take also into account that, according to the VTL consistency rules, the agency of the result of a ,,Transformation,, must be the same as its ,,TransformationScheme,,, therefore the agency-id can be omitted for all the results (left part of ,,Transformation,, statements). 1352 1352 * As for the **maintainedobject-id**, this is essential in some cases while in other cases it can be omitted: o if the referenced artefact is a ,,Dataflow,,, which is a maintainable class, the maintainedobject-id is the dataflow-id and obviously cannot be omitted; 1353 1353 ** if the referenced artefact is a Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, which are not maintainable and belong to the ,,DataStructure,, maintainable class, the maintainedobject-id is the dataStructure-id and can be omitted, given that these components are always invoked within the invocation of a ,,Dataflow,,, whose dataStructure-id can be deduced from the 1354 1354 ... ... @@ -1359,7 +1359,7 @@ 1359 1359 ** if the referenced artefact is a ,,Codelist, ,,which is a maintainable class, the maintainedobject-id is the ,,codelist-id,, and obviously cannot be omitted. 1360 1360 * When the maintainedobject-id is omitted, the **maintainedobject-version** is omitted too. When the maintainedobject-id is not omitted and the maintainedobject-version is omitted, the version 1.0 is assumed by default.,, ,, 1361 1361 * As said, the **container-object-id** does not apply to the classes that can be referenced in VTL transformations, therefore is not present in their URN 1362 -* The **object-id** does not exist for the artefacts belonging to the ,,Dataflow, ,,,,ConceptScheme,, and ,,Codelist,, classes, while it exists and cannot be omitted for the artefacts belonging to the classes Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute and Concept, as for1286 +* The **object-id** does not exist for the artefacts belonging to the ,,Dataflow, ConceptScheme,, and ,,Codelist,, classes, while it exists and cannot be omitted for the artefacts belonging to the classes Dimension, MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute and Concept, as for 1363 1363 1364 1364 them the object-id is the main identifier of the artefact 1365 1365 ... ... @@ -1371,15 +1371,15 @@ 1371 1371 1372 1372 ‘urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF2(1.0)’ 1373 1373 1374 -by omitting all the non-essential parts would become simply: 1298 +by omitting all the non-essential parts would become simply: 1375 1375 1376 1376 DFR := DF1 + DF2 1377 1377 1378 -The references to the ,,Codelists,, can be simplified similarly. For example, given the non-abbreviated reference to the ,,Codelist,, AG:CL_FREQ(1.0), which is[[^^~[14~]^^>>path:#_ftn14]]: 1302 +The references to the ,,Codelists,, can be simplified similarly. For example, given the non-abbreviated reference to the ,,Codelist,, AG:CL_FREQ(1.0), which is[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[14~]^^>>path:#_ftn14]](%%): 1379 1379 1380 1380 ‘urn:sdmx:org.sdmx.infomodel.codelist.Codelist=AG:CL_FREQ(1.0)’ 1381 1381 1382 -if the ,,Codelist,, is referenced from a ruleset scheme belonging to the agency AG, omitting all the optional parts, the abbreviated reference would become simply[[^^~[15~]^^>>path:#_ftn15]]: 1306 +if the ,,Codelist,, is referenced from a ruleset scheme belonging to the agency AG, omitting all the optional parts, the abbreviated reference would become simply[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[15~]^^>>path:#_ftn15]](%%): 1383 1383 1384 1384 CL_FREQ 1385 1385 ... ... @@ -1389,7 +1389,7 @@ 1389 1389 1390 1390 SECTOR 1391 1391 1392 -For example, the transformation for renaming the component SECTOR of the dataflow DF1 into SEC can be written as[[^^~[16~]^^>>path:#_ftn16]]: 1316 +For example, the transformation for renaming the component SECTOR of the dataflow DF1 into SEC can be written as[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[16~]^^>>path:#_ftn16]](%%): 1393 1393 1394 1394 ‘DFR(1.0)’ := ‘DF1(1.0)’ [rename SECTOR to SEC] 1395 1395 ... ... @@ -1423,9 +1423,9 @@ 1423 1423 1424 1424 The VTL Rulesets have a signature, in which the Value Domains or the Variables on which the Ruleset is defined are declared, and a body, which contains the rules. 1425 1425 1426 -In the signature, given the mapping between VTL and SDMX better described in the following paragraphs, a reference to a VTL Value Domain becomes a reference to a SDMX Codelist or to a SDMX ConceptScheme (for SDMX measure dimensions), while a reference to a VTL Represented Variable becomes a reference to a SDMX Concept, assuming for it a definite representation[[^^~[17~]^^>>path:#_ftn17]]. 1350 +In the signature, given the mapping between VTL and SDMX better described in the following paragraphs, a reference to a VTL Value Domain becomes a reference to a SDMX Codelist or to a SDMX ConceptScheme (for SDMX measure dimensions), while a reference to a VTL Represented Variable becomes a reference to a SDMX Concept, assuming for it a definite representation[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[17~]^^>>path:#_ftn17]](%%). 1427 1427 1428 -In general, for referencing SDMX Codelists and Concepts, the conventions described in the previous paragraphs apply. In the Ruleset syntax, the elements that reference SDMX artefacts are called “valueDomain” and “variable” for the Datapoint Rulesets and “ruleValueDomain”, “ruleVariable”, “condValueDomain” “condVariable” for the Hierarchical Rulesets). The syntax of the Ruleset signature allows also to define aliases of the elements above, these aliases are valid only within the specific ruleset definition statement and cannot be mapped to SDMX.[[^^~[18~]^^>>path:#_ftn18]] 1352 +In general, for referencing SDMX Codelists and Concepts, the conventions described in the previous paragraphs apply. In the Ruleset syntax, the elements that reference SDMX artefacts are called “valueDomain” and “variable” for the Datapoint Rulesets and “ruleValueDomain”, “ruleVariable”, “condValueDomain” “condVariable” for the Hierarchical Rulesets). The syntax of the Ruleset signature allows also to define aliases of the elements above, these aliases are valid only within the specific ruleset definition statement and cannot be mapped to SDMX.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[18~]^^>>path:#_ftn18]](%%) 1429 1429 1430 1430 In the body of the Rulesets, the Codes and in general all the Values can be written without any other specification, because the artefact which the Values are referred (Codelist, ConceptScheme, Concept) to can be deduced from the Ruleset signature. 1431 1431 ... ... @@ -1439,15 +1439,15 @@ 1439 1439 1440 1440 Every time a SDMX object is referenced in a VTL Transformation as an input operand, there is the need to generate a VTL definition of the object, so that the VTL operations can take place. This can be made starting from the SDMX definition and applying a SDMX-VTL mapping method in the direction from SDMX to VTL. The possible mapping methods from SDMX to VTL are described in the following paragraphs and are conceived to allow the automatic deduction of the VTL definition of the object from the knowledge of the SDMX definition. 1441 1441 1442 -In the opposite direction, every time an object calculated by means of VTL must be treated as a SDMX object (for example for exchanging it through SDMX), there is the need of a SDMX definition of the object, so that the SDMX operations can take place. The SDMX definition is needed for the VTL objects for which a SDMX use is envisaged[[^^~[19~]^^>>path:#_ftn19]]. 1366 +In the opposite direction, every time an object calculated by means of VTL must be treated as a SDMX object (for example for exchanging it through SDMX), there is the need of a SDMX definition of the object, so that the SDMX operations can take place. The SDMX definition is needed for the VTL objects for which a SDMX use is envisaged[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[19~]^^>>path:#_ftn19]](%%). 1443 1443 1444 1444 The mapping methods from VTL to SDMX are described in the following paragraphs as well, however they do not allow the complete SDMX definition to be automatically deduced from the VTL definition, more than all because the former typically contains additional information in respect to the latter. For example, the definition of a SDMX DSD includes also some mandatory information not available in VTL (like the concept scheme to which the SDMX components refer, the assignmentStatus and attributeRelationship for the DataAttributes and so on). Therefore the mapping methods from VTL to SDMX provide only a general guidance for generating SDMX definitions properly starting from the information available in VTL, independently of how the SDMX definition it is actually generated (manually, automatically or part and part). 1445 1445 1446 1446 === 10.3.2 General mapping of VTL and SDMX data structures === 1447 1447 1448 -This section makes reference to the VTL “Model for data and their structure”[[^^~[20~]^^>>path:#_ftn20]] and the correspondent SDMX “Data Structure Definition”[[^^~[21~]^^>>path:#_ftn21]]. 1372 +This section makes reference to the VTL “Model for data and their structure”[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[20~]^^>>path:#_ftn20]](%%) and the correspondent SDMX “Data Structure Definition”[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[21~]^^>>path:#_ftn21]](%%). 1449 1449 1450 -The main type of artefact that the VTL can manipulate is the VTL Data Set, which in general is mapped to the SDMX Dataflow. This means that a VTL Transformation, in the SDMX context, expresses the algorithm for calculating a derived Dataflow starting from some already existing Dataflows (either collected or derived).[[^^~[22~]^^>>path:#_ftn22]] 1374 +The main type of artefact that the VTL can manipulate is the VTL Data Set, which in general is mapped to the SDMX Dataflow. This means that a VTL Transformation, in the SDMX context, expresses the algorithm for calculating a derived Dataflow starting from some already existing Dataflows (either collected or derived).[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[22~]^^>>path:#_ftn22]](%%) 1451 1451 1452 1452 While the VTL Transformations are defined in term of Dataflow definitions, they are assumed to be executed on instances of such Dataflows, provided at runtime to the VTL engine (the mechanism for identifying the instances to be processed are not part of the VTL specifications and depend on the implementation of the VTL-based systems). As already said, the SDMX Datasets are instances of SDMX Dataflows, therefore a VTL Transformation defined on some SDMX Dataflows can be applied on some corresponding SDMX Datasets. 1453 1453 ... ... @@ -1457,7 +1457,7 @@ 1457 1457 1458 1458 SDMX DimensionComponent can be a Dimension, a TimeDimension or a MeasureDimension. Correspondingly, in the SDMX implementation of the VTL, the VTL Identifiers can be (optionally) distinguished in three sub-classes (Simple Identifier, Time Identifier, Measure Identifier) even if such a distinction is not evidenced in the VTL IM. 1459 1459 1460 -However, a VTL Data Structure can have any number of Identifiers, Measures and Attributes, while a SDMX 2.1 DataStructureDefinition can have any number of Dimensions and DataAttributes but just one PrimaryMeasure[[^^~[23~]^^>>path:#_ftn23]]. This is due to a difference between SDMX 2.1 and VTL in the possible representation methods of the data that contain more measures. 1384 +However, a VTL Data Structure can have any number of Identifiers, Measures and Attributes, while a SDMX 2.1 DataStructureDefinition can have any number of Dimensions and DataAttributes but just one PrimaryMeasure[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[23~]^^>>path:#_ftn23]](%%). This is due to a difference between SDMX 2.1 and VTL in the possible representation methods of the data that contain more measures. 1461 1461 1462 1462 As for SDMX, because the data structure cannot contain more than one measure component (i.e., the primaryMeasure), the representation of data having more measures is possible only by means of a particular dimension, called MeasureDimension, which is aimed at containing the name of the measure concepts, so that for each observation the value contained in the PrimaryMeasure component is the value of the measure concept reported in the MeasureDimension component. 1463 1463 ... ... @@ -1547,7 +1547,7 @@ 1547 1547 1548 1548 This mapping method cannot be applied for SDMX 2.1 if the VTL data structure has more than one measure component, given that the SDMX 2.1 DataStructureDefinition allows just one measure component (the 1549 1549 1550 -PrimaryMeasure). In this case it becomes mandatory to specify a different 1958 mapping method through the VtlMappingScheme and VtlDataflowMapping 1959 classes.[[^^~[24~]^^>>path:#_ftn24]] 1474 +PrimaryMeasure). In this case it becomes mandatory to specify a different 1958 mapping method through the VtlMappingScheme and VtlDataflowMapping 1959 classes.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[24~]^^>>path:#_ftn24]](%%) 1551 1551 1552 1552 1960 Please note that the VTL measures can have any name while in SDMX 2.1 the 1961 MeasureComponent has the mandatory name “obs_value”, therefore the name of the VTL measure name must become “obs_value” in SDMX 2.1. 1553 1553 ... ... @@ -1664,15 +1664,15 @@ 1664 1664 1665 1665 The VtlMappingScheme is a container for zero or more VtlDataflowMapping (besides possible mappings to artefacts other than dataflows). 1666 1666 1667 -=== 10.3.6 Mapping dataflow subsets to distinct VTL data sets[[^^**~[25~]**^^>>path:#_ftn25]] 1591 +=== 10.3.6 Mapping dataflow subsets to distinct VTL data sets[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^**~[25~]**^^>>path:#_ftn25]](%%) === 1668 1668 1669 1669 Until now it as been assumed to map one SMDX Dataflow to one VTL dataset and vice-versa. This mapping one-to-one is not mandatory according to VTL because a VTL data set is meant to be a set of observations (data points) on a logical plane, having the same logical data structure and the same general meaning, independently of the possible physical representation or storage (see VTL 2.0 User Manual page 1670 1670 1671 1671 24), therefore a SDMX Dataflow can be seen either as a unique set of data observations (corresponding to one VTL data set) or as the union of many sets of data observations (each one corresponding to a distinct VTL data set). 1672 1672 1673 -As a matter of fact, in some cases it can be useful to define VTL operations involving definite parts of a SDMX Dataflow instead than the whole.[[^^~[26~]^^>>path:#_ftn26]] 1597 +As a matter of fact, in some cases it can be useful to define VTL operations involving definite parts of a SDMX Dataflow instead than the whole.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[26~]^^>>path:#_ftn26]](%%) 1674 1674 1675 -Therefore, in order to make the coding of VTL operations simpler when applied on parts of SDMX Dataflows, it is allowed to map distinct parts of a SDMX Dataflow to distinct VTL data sets according to the following rules and conventions. This kind of mapping is possible both from SDMX to VTL and from VTL to SDMX, as better explained below.[[^^~[27~]^^>>path:#_ftn27]] 1599 +Therefore, in order to make the coding of VTL operations simpler when applied on parts of SDMX Dataflows, it is allowed to map distinct parts of a SDMX Dataflow to distinct VTL data sets according to the following rules and conventions. This kind of mapping is possible both from SDMX to VTL and from VTL to SDMX, as better explained below.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[27~]^^>>path:#_ftn27]](%%) 1676 1676 1677 1677 Given a SDMX Dataflow and some predefined Dimensions of its 1678 1678 ... ... @@ -1684,14 +1684,14 @@ 1684 1684 1685 1685 In practice, this kind mapping is obtained like follows: 1686 1686 1687 -* For a given SDMX dataflow, the user (VTL definer) declares the dimension components on which the mapping will be based, in a given order.[[^^~[28~]^^>>path:#_ftn28]] Following the example above, imagine that the user declares the dimensions INDICATOR and COUNTRY. 1611 +* For a given SDMX dataflow, the user (VTL definer) declares the dimension components on which the mapping will be based, in a given order.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[28~]^^>>path:#_ftn28]](%%) Following the example above, imagine that the user declares the dimensions INDICATOR and COUNTRY. 1688 1688 * The VTL dataset is given a name using a special notation also called “ordered concatenation” and composed of the following parts: 1689 1689 ** The reference to the SDMX dataflow (expressed according to the rules described in the previous paragraphs, i.e. URN, abbreviated 1690 1690 1691 -URN or another alias); for example DF(1.0); o a slash (“/”) as a separator; [[^^~[29~]^^>>path:#_ftn29]] 1615 +URN or another alias); for example DF(1.0); o a slash (“/”) as a separator; [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[29~]^^>>path:#_ftn29]] 1692 1692 1693 1693 * 1694 -** The reference to a specific part of the SDMX dataflow above, expressed as the concatenation of the values that the SDMX dimensions declared above must have, separated by dots (“.”) and written in the order in which these dimensions are defined[[^^~[30~]^^>>path:#_ftn30]] . For example POPULATION.USA would mean that such a VTL dataset is mapped to the SDMX observations for which the dimension //INDICATOR// is equal to POPULATION and the dimension //COUNTRY// is equal to USA. 1618 +** The reference to a specific part of the SDMX dataflow above, expressed as the concatenation of the values that the SDMX dimensions declared above must have, separated by dots (“.”) and written in the order in which these dimensions are defined[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[30~]^^>>path:#_ftn30]](%%) . For example POPULATION.USA would mean that such a VTL dataset is mapped to the SDMX observations for which the dimension //INDICATOR// is equal to POPULATION and the dimension //COUNTRY// is equal to USA. 1695 1695 1696 1696 In the VTL transformations, this kind of dataset name must be referenced between single quotes because the slash (“/”) is not a regular character according to the VTL rules. 1697 1697 ... ... @@ -1709,7 +1709,7 @@ 1709 1709 1710 1710 Let us now analyse the different meaning of this kind of mapping in the two mapping directions, i.e. from SDMX to VTL and from VTL to SDMX. 1711 1711 1712 -As already said, the mapping from SDMX to VTL happens when the VTL datasets are operand of VTL transformations, instead the mapping from VTL to SDMX happens when the VTL datasets are result of VTL transformations[[^^~[31~]^^>>path:#_ftn31]] and need to be treated as SDMX objects. This kind of mapping can be applied independently in the two directions and the Dimensions on which the mapping is based can be different in the two directions: these Dimensions are defined in the ToVtlSpaceKey and in the FromVtlSpaceKey classes respectively. 1636 +As already said, the mapping from SDMX to VTL happens when the VTL datasets are operand of VTL transformations, instead the mapping from VTL to SDMX happens when the VTL datasets are result of VTL transformations[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[31~]^^>>path:#_ftn31]](%%) and need to be treated as SDMX objects. This kind of mapping can be applied independently in the two directions and the Dimensions on which the mapping is based can be different in the two directions: these Dimensions are defined in the ToVtlSpaceKey and in the FromVtlSpaceKey classes respectively. 1713 1713 1714 1714 First, let us see what happens in the mapping direction from SDMX to VTL, i.e. when parts of a SDMX dataflow (e.g. DF1(1.0)) need to be mapped to distinct VTL datasets that are operand of some VTL transformations. 1715 1715 ... ... @@ -1719,7 +1719,7 @@ 1719 1719 1720 1720 //COUNTRYvalue//. For example, the VTL dataset ‘DF1(1.0)/POPULATION.USA’ would contain all the observations of DF1(1.0) having INDICATOR = POPULATION and COUNTRY = USA. 1721 1721 1722 -In order to obtain the data structure of these VTL datasets from the SDMX one, it is assumed that the SDMX dimensions on which the mapping is based are dropped, i.e. not maintained in the VTL data structure; this is possible because their values are fixed for each one of the invoked VTL datasets[[^^~[32~]^^>>path:#_ftn32]]. After that, the mapping method from SDMX to VTL specified for the dataflow DF1(1.0) is applied (i.e. basic, pivot …). 1646 +In order to obtain the data structure of these VTL datasets from the SDMX one, it is assumed that the SDMX dimensions on which the mapping is based are dropped, i.e. not maintained in the VTL data structure; this is possible because their values are fixed for each one of the invoked VTL datasets[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[32~]^^>>path:#_ftn32]](%%). After that, the mapping method from SDMX to VTL specified for the dataflow DF1(1.0) is applied (i.e. basic, pivot …). 1723 1723 1724 1724 In the example above, for all the datasets of the kind 1725 1725 ... ... @@ -1739,7 +1739,7 @@ 1739 1739 1740 1740 … … … 1741 1741 1742 -In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX dataflow to different VTL datasets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a dataflow. [[^^~[33~]^^>>path:#_ftn33]] 1666 +In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX dataflow to different VTL datasets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a dataflow. [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[33~]^^>>path:#_ftn33]] 1743 1743 1744 1744 In the direction from SDMX to VTL it is allowed to omit the value of one or more Dimensions on which the mapping is based, but maintaining all the separating dots (therefore it may happen to find two or more consecutive dots and dots in the beginning or in the end). The absence of value means that for the corresponding Dimension all the values are kept and the Dimension is not dropped. 1745 1745 ... ... @@ -1762,12 +1762,12 @@ 1762 1762 1763 1763 For example, let us assume that the VTL programmer wants to calculate the SDMX dataflow DF2(1.0) having the Dimensions TIME_PERIOD, INDICATOR, and COUNTRY and that such a programmer finds it convenient to calculate separately the parts of DF2(1.0) that have different combinations of values for INDICATOR and COUNTRY: 1764 1764 1765 -* each part is calculated as a VTL derived dataset, result of a dedicated VTL transformation; [[^^~[34~]^^>>path:#_ftn34]] 1766 -* the data structure of all these VTL datasets has the TIME_PERIOD identifier and does not have the INDICATOR and COUNTRY identifiers.[[^^~[35~]^^>>path:#_ftn35]] 1689 +* each part is calculated as a VTL derived dataset, result of a dedicated VTL transformation; [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[34~]^^>>path:#_ftn34]](%%) 1690 +* the data structure of all these VTL datasets has the TIME_PERIOD identifier and does not have the INDICATOR and COUNTRY identifiers.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[35~]^^>>path:#_ftn35]] 1767 1767 1768 -Under these hypothesis, such derived VTL datasets can be mapped to DF2(1.0) by declaring the Dimensions INDICATOR and COUNTRY as mapping dimensions[[^^~[36~]^^>>path:#_ftn36]]. 1692 +Under these hypothesis, such derived VTL datasets can be mapped to DF2(1.0) by declaring the Dimensions INDICATOR and COUNTRY as mapping dimensions[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[36~]^^>>path:#_ftn36]](%%). 1769 1769 1770 -The corresponding VTL transformations, assuming that the result needs to be persistent, would be of this kind:^^ ^^[[^^~[37~]^^>>path:#_ftn37]] 1694 +The corresponding VTL transformations, assuming that the result needs to be persistent, would be of this kind:^^ ^^[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[37~]^^>>path:#_ftn37]] 1771 1771 1772 1772 ‘DF2(1.0)///INDICATORvalue//.//COUNTRYvalue//’ <- expression 1773 1773 ... ... @@ -1789,19 +1789,19 @@ 1789 1789 As said, it is assumed that these VTL derived datasets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 1790 1790 1791 1791 |((( 1792 - //VTL dataset 1716 + //VTL dataset // 1793 1793 1794 1794 1795 1795 )))|(% colspan="2" %)//INDICATOR value //|(% colspan="2" %)//COUNTRY value// 1796 -|‘DF2(1.0)/GDPPERCAPITA.USA’ 1720 +|‘DF2(1.0)/GDPPERCAPITA.USA’ |GDPPERCAPITA| | |USA 1797 1797 |((( 1798 1798 ‘DF2(1.0)/GDPPERCAPITA.CANADA’ 1799 1799 1800 1800 … … … 1801 1801 )))|GDPPERCAPITA| | |CANADA 1802 -|‘DF2(1.0)/POPGROWTH.USA’ 1726 +|‘DF2(1.0)/POPGROWTH.USA’ |POPGROWTH | | |USA 1803 1803 |((( 1804 -‘DF2(1.0)/POPGROWTH.CANADA’ 1728 +‘DF2(1.0)/POPGROWTH.CANADA’ 1805 1805 1806 1806 … … … 1807 1807 )))|POPGROWTH | | |CANADA ... ... @@ -1834,9 +1834,9 @@ 1834 1834 1835 1835 …); 1836 1836 1837 -In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent data sets are united and give the final result DF2(1.0)[[^^~[38~]^^>>path:#_ftn38]], which can be mapped one-to-one to the homonymous SDMX dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. 1761 +In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent data sets are united and give the final result DF2(1.0)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[38~]^^>>path:#_ftn38]](%%), which can be mapped one-to-one to the homonymous SDMX dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. 1838 1838 1839 -Therefore, mapping different VTL datasets having the same data structure to different parts of a SDMX dataflow, i.e. in the direction from VTL to SDMX, through the ordered concatenation notation is equivalent to a proper use of the operators “calc” and “union” on such datasets. [[^^~[39~]^^>>path:#_ftn39]][[^^~[40~]^^>>path:#_ftn40]] 1763 +Therefore, mapping different VTL datasets having the same data structure to different parts of a SDMX dataflow, i.e. in the direction from VTL to SDMX, through the ordered concatenation notation is equivalent to a proper use of the operators “calc” and “union” on such datasets. [[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[39~]^^>>path:#_ftn39]](%%)[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[40~]^^>>path:#_ftn40]] 1840 1840 1841 1841 It is worth noting that in the direction from VTL to SDMX it is mandatory to specify the value for every Dimension on which the mapping is based (in other word, in the name of the calculated VTL dataset is not possible to omit the value of some of the Dimensions). 1842 1842 ... ... @@ -1859,7 +1859,7 @@ 1859 1859 ))) 1860 1860 |**Code**|**Code** (for enumerated Dimension, PrimaryMeasure, DataAttribute) or **Concept** (for MeasureDimension) 1861 1861 |**Described Value Domain**|((( 1862 -non-enumerated** 1786 +non-enumerated** Representation** 1863 1863 1864 1864 (having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 1865 1865 ))) ... ... @@ -1885,7 +1885,7 @@ 1885 1885 1886 1886 Domain) is not identifiable. As a consequence, the definition of the VTL rulesets, which in VTL can refer either to enumerated or non-enumerated value domains, in SDMX can refer only to enumerated Value Domains (i.e. to SDMX Codelists). 1887 1887 1888 -As for the mapping between VTL variables and SDMX Concepts, it should be noted that these artefacts do not coincide perfectly. In fact, the VTL variables are represented variables, defined always on the same Value Domain (“Representation” in SDMX) independently of the data set / data structure in which they appear[[^^~[41~]^^>>path:#_ftn41]], while the SDMX Concepts can have different Representations in different DataStructures.[[^^~[42~]^^>>path:#_ftn42]] This means that one SDMX Concept can correspond to many VTL Variables, one for each representation the Concept has. 1812 +As for the mapping between VTL variables and SDMX Concepts, it should be noted that these artefacts do not coincide perfectly. In fact, the VTL variables are represented variables, defined always on the same Value Domain (“Representation” in SDMX) independently of the data set / data structure in which they appear[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[41~]^^>>path:#_ftn41]](%%), while the SDMX Concepts can have different Representations in different DataStructures.[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[42~]^^>>path:#_ftn42]](%%) This means that one SDMX Concept can correspond to many VTL Variables, one for each representation the Concept has. 1889 1889 1890 1890 Therefore, it is important to be aware that some VTL operations (for example the binary operations at data set level) are consistent only if the components having the same names in the operated VTL data sets have also the same representation (i.e. the same Value Domain as for VTL). For example, it is possible to obtain correct results from the VTL expression 1891 1891 ... ... @@ -1934,7 +1934,7 @@ 1934 1934 1935 1935 The opposite conversion, i.e. from VTL to SDMX, happens when a VTL result, i.e. a VTL data set output of a transformation, must become a SDMX artefact (or part of it). The values of the VTL result must be converted into the desired (SDMX) external representations (data types) of the SDMX artefact. 1936 1936 1937 -=== 10.4.3 Mapping SDMX data types to VTL basic scalar types 1861 +=== 10.4.3 Mapping SDMX data types to VTL basic scalar types === 1938 1938 1939 1939 The following table describes the default mapping for converting from the SDMX data types to the VTL basic scalar types. 1940 1940 ... ... @@ -2001,7 +2001,7 @@ 2001 2001 |((( 2002 2002 **Boolean ** 2003 2003 2004 -(corresponds to the XML Schema xs:boolean datatype; support the mathematical concept of binary-valued logic: {true, false}) 1928 +(corresponds to the XML Schema xs:boolean datatype; support the mathematical concept of binary-valued logic: {true, false}) 2005 2005 )))|**boolean** 2006 2006 |((( 2007 2007 **URI ** ... ... @@ -2144,7 +2144,7 @@ 2144 2144 2145 2145 When VTL takes in input SDMX artefacts, it is assumed that a type conversion according to the table above always happens. In case a different VTL basic scalar type is desired, it can be achieved in the VTL program taking in input the default VTL basic scalar type above and applying to it the VTL type conversion features (see the implicit and explicit type conversion and the “cast” operator in the VTL Reference Manual). 2146 2146 2147 -=== 10.4.4 Mapping VTL basic scalar types to SDMX data types 2071 +=== 10.4.4 Mapping VTL basic scalar types to SDMX data types === 2148 2148 2149 2149 The following table describes the default conversion from the VTL basic scalar types to the SDMX data types . 2150 2150 ... ... @@ -2174,12 +2174,12 @@ 2174 2174 “true” or “false” 2175 2175 ))) 2176 2176 2177 -**Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types** 2178 2178 2179 -In case a different default conversion is desired, it can be achieved through the 2180 2180 2181 - CustomTypeSchemeand CustomTypeartefacts(see alsothe sectionTransformationsand Expressionsof theSDMX information model).2103 +**Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types** 2182 2182 2105 +In case a different default conversion is desired, it can be achieved through the CustomTypeScheme and CustomType artefacts (see also the section Transformations and Expressions of the SDMX information model). 2106 + 2183 2183 The custom output formats can be specified by means of the VTL formatting mask described in the section “Type Conversion and Formatting Mask” of the VTL Reference Manual. Such a section describes the masks for the VTL basic scalar types “number”, “integer”, “date”, “time”, “time_period” and “duration” and gives examples. As for the types “string” and “boolean” the VTL conventions are extended with some other special characters as described in the following table. 2184 2184 2185 2185 |(% colspan="2" %)**VTL special characters for the formatting masks** ... ... @@ -2230,7 +2230,7 @@ 2230 2230 |N|fixed number of digits used in the preceding textual representation of the month or the day 2231 2231 | | 2232 2232 2233 -The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion[[^^~[43~]^^>>path:#_ftn43]]. 2157 +The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^~[43~]^^>>path:#_ftn43]](%%). 2234 2234 2235 2235 === 10.4.5 Null Values === 2236 2236 ... ... @@ -2262,12 +2262,18 @@ 2262 2262 2263 2263 For implementing an SDMX compliant Web Service the standardised WSDL file should be used that describes the expected request/response structure. The request message of the operation contains a wrapper element (e.g. “GetGenericData”) that wraps a tag called “GenericDataQuery”, which is the actual SDMX query XML message that contains the query to be processed by the Web Service. In the same way the response is formulated in a wrapper element “GetGenericDataResponse”. 2264 2264 2265 -As defined in the SOAP specification, the root element of a SOAP message is the Envelope, which contains an optional Header and a mandatory Body. These are illustrated below along with the Body contents according to the WSDL: 2189 +As defined in the SOAP specification, the root element of a SOAP message is the Envelope, which contains an optional Header and a mandatory Body. These are illustrated below along with the Body contents according to the WSDL: 2266 2266 2191 +[[image:1747854006117-843.png]] 2192 + 2267 2267 The problem that initiated the present analysis refers to the difference in the way SOAP requests are when trying to implement the aforementioned Web Service in .NET framework. 2268 2268 2269 2269 Building such a Web Service using the .NET framework is done by exposing a method (i.e. the getGenericData in the example) with an XML document argument (lets name it “Query”). **The difference that appears in Microsoft .Net implementations is that there is a need for an extra XML container around the SDMX GenericDataQuery.** This is the expected behavior since the framework is let to publish automatically the Web Service as a remote procedure call, thus wraps each parameter into an extra element. The .NET request is illustrated below: 2270 2270 2197 +[[image:1747854039499-443.png]] 2198 + 2199 +[[image:1747854067769-691.png]] 2200 + 2271 2271 Furthermore this extra element is also inserted in the automatically generated WSDL from the framework. Therefore this particularity requires custom clients for the .NET Web Services that is not an interoperable solution. 2272 2272 2273 2273 == 11.2 Solution == ... ... @@ -2288,20 +2288,30 @@ 2288 2288 2289 2289 To understand how the **XmlAnyElement** attribute works we present the following two web methods: 2290 2290 2291 - In thismethod the **input** parameter is decorated with the **XmlAnyElement** parameter. This is a hint that this parameter will be de-serialized from an **xsd:any** element.Since the attribute is notpassed any parameters, it means that the entire XML element for this parameter in the SOAP message will be in the Infoset that is represented by this **XmlElement** parameter.2221 +[[image:1747854096778-844.png]] 2292 2292 2293 - Thedifferencebetween the two is that for thefirst method,**SubmitXml**,the2223 +In this method the **input** parameter is decorated with the **XmlAnyElement** parameter. This is a hint that this parameter will be de-serialized from an **xsd:any** element. Since the attribute is not passed any parameters, it means that the entire XML element for this parameter in the SOAP message will be in the Infoset that is represented by this **XmlElement** parameter. 2294 2294 2295 - XmlSerializer will expect an element named **input** to be an immediate child of the **SubmitXml** element in the SOAP body. The second method, **SubmitXmlAny**, will not care what the name of the child of the **SubmitXmlAny** element is. It will plugwhatever XML is included into the input parameter.The message style from ASP.NET Helpfor the two methods is shownbelow. First we look at the message for the method without the **XmlAnyElement** attribute.2225 +[[image:1747854127303-270.png]] 2296 2296 2227 +The difference between the two is that for the first method, **SubmitXml**, the XmlSerializer will expect an element named **input** to be an immediate child of the **SubmitXml** element in the SOAP body. The second method, **SubmitXmlAny**, will not care what the name of the child of the **SubmitXmlAny** element is. It will plug whatever XML is included into the input parameter. The message style from ASP.NET Help for the two methods is shown below. First we look at the message for the method without the **XmlAnyElement** attribute. 2228 + 2229 +[[image:1747854163928-581.png]] 2230 + 2297 2297 Now we look at the message for the method that uses the **XmlAnyElement** attribute. 2298 2298 2233 +[[image:1747854190641-364.png]] 2234 + 2235 +[[image:1747854236732-512.png]] 2236 + 2299 2299 The method decorated with the **XmlAnyElement** attribute has one fewer wrapping elements. Only an element with the name of the method wraps what is passed to the **input** parameter. 2300 2300 2301 -For more information please consult: >>url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]][[->>url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]][[us/library/aa480498.aspx>>url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]][[url:http://msdn.microsoft.com/en-us/library/aa480498.aspx]]2239 +For more information please consult: [[http:~~/~~/msdn.microsoft.com/en-us/library/aa480498.aspx>>http://msdn.microsoft.com/en-us/library/aa480498.aspx]] 2302 2302 2303 2303 Furthermore at this point the problem with the different requests has been solved. However there is still the difference in the produced WSDL that has to be taken care. The automatic generated WSDL now doesn’t insert the extra element, but defines the content of the operation wrapper element as “xsd:any” type. 2304 2304 2243 +[[image:1747854286398-614.png]] 2244 + 2305 2305 Without a common WSDL still the solution doesn’t enforce interoperability. In order to 2306 2306 2307 2307 “fix” the WSDL, there two approaches. The first is to intervene in the generation process. This is a complicated approach, compared to the second approach, which overrides the generation process and returns the envisioned WSDL for the SDMX Web Service. ... ... @@ -2314,16 +2314,27 @@ 2314 2314 2315 2315 In the context of the SDMX Web Service, applying the above solution translates into the following: 2316 2316 2257 +[[image:1747854385465-132.png]] 2258 + 2317 2317 The SOAP request/response will then be as follows: 2318 2318 2319 2319 **GenericData Request** 2320 2320 2263 +[[image:1747854406014-782.png]] 2264 + 2321 2321 **GenericData Response** 2322 2322 2267 +[[image:1747854424488-855.png]] 2268 + 2323 2323 For overriding the automatically produced WSDL, in the solution explorer right click the project and select “Add” -> “New item…”. Then select the “Global Application Class”. This will create “.asax” class file in which the following code should replace the existing empty method: 2324 2324 2271 +[[image:1747854453895-524.png]] 2272 + 2273 +[[image:1747854476631-125.png]] 2274 + 2325 2325 The SDMX_WSDL.wsdl should reside in the in the root directory of the application. After applying this solution the returned WSDL is the envisioned. Thus in the request message definition contains: 2326 2326 2277 +[[image:1747854493363-776.png]] 2327 2327 2328 2328 ---- 2329 2329
- 1747854006117-843.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +18.7 KB - Content
- 1747854039499-443.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +3.0 KB - Content
- 1747854067769-691.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +21.4 KB - Content
- 1747854096778-844.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +10.6 KB - Content
- 1747854127303-270.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +12.1 KB - Content
- 1747854163928-581.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +29.8 KB - Content
- 1747854190641-364.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +31.5 KB - Content
- 1747854236732-512.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +2.0 KB - Content
- 1747854286398-614.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content
- 1747854385465-132.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content
- 1747854406014-782.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content
- 1747854424488-855.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content
- 1747854453895-524.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content
- 1747854476631-125.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content
- 1747854493363-776.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +55.0 KB - Content