Changes for page 12 Validation and Transformation Language (VTL)
Last modified by Helena K. on 2026/06/10 10:09
Summary
-
Page properties (2 modified, 0 added, 0 removed)
-
Objects (0 modified, 1 added, 1 removed)
Details
- Page properties
-
- Tags
-
... ... @@ -1,1 +1,0 @@ 1 -Artefact|Attribute|Code|Code list|Component|Concept scheme|Currency|Data set|Data structure definition|Dataflow|Dimension|Facet|Maintainable artefact|Measure|Nameable artefact|Representation|SDMX Information Model|Statistical data and metadata exchange|Structural metadata|Validation and transformation language - Content
-
... ... @@ -18,7 +18,7 @@ 18 18 19 19 This section does not explain the VTL language or any of the content published in the VTL guides. Rather, this is a description of how the VTL can be used in the SDMX context and applied to SDMX artefacts. 20 20 21 -== 12.2 References to SDMX artefacts from VTL statements == 21 +== 12.2 References to SDMX artefacts from VTL statements == 22 22 23 23 === 12.2.1 Introduction === 24 24 ... ... @@ -78,12 +78,8 @@ 78 78 * if the artefact is a Dimension, TimeDimension, Measure or DataAttribute (the object-id is the name of one of the artefacts above, which are data structure components) 79 79 * if the artefact is a Concept (the object-id is the name of the Concept) 80 80 81 +For example, by using the URN, the VTL Transformation that sums two SDMX Dataflows DF1 and DF2 and assigns the result to a third persistent Dataflow DFR, assuming that DF1, DF2 and DFR are the maintainedobject-id of the three Dataflows, that their version is 1.0.0 and their Agency is AG, would be written as{{footnote}}Since these references to SDMX objects include non-permitted characters as per the VTL ID notation, they need to be included between single quotes, according to the VTL rules for irregular names.{{/footnote}}: 81 81 82 - 83 -For example, by using the URN, the VTL Transformation that sums two SDMX Dataflows DF1 and DF2 and assigns the result to a third persistent Dataflow DFR, assuming that DF1, DF2 and DFR are the maintainedobject-id of the three Dataflows, that their version is 1.0.0 and their Agency is AG, would be written as 84 - 85 -{{footnote}}Since these references to SDMX objects include non-permitted characters as per the VTL ID notation, they need to be included between single quotes, according to the VTL rules for irregular names.{{/footnote}}: 86 - 87 87 > 'urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DFR(1.0.0)' <- 88 88 > 'urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF1(1.0.0)' + 89 89 > 'urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=AG:DF2(1.0.0)' ... ... @@ -110,8 +110,6 @@ 110 110 * As said, the container-object-id does not apply to the classes that can be referenced in VTL Transformations, therefore is not present in their URN 111 111 * The object-id does not exist for the artefacts belonging to the Dataflow, and Codelist classes, while it exists and cannot be omitted for the artefacts belonging to the classes Dimension, TimeDimension, Measure, DataAttribute and Concept, as for them the object-id is the main identifier of the artefact 112 112 113 - 114 - 115 115 The simplified object identifier is obtained by omitting all the first part of the URN, including the special characters, till the first part not omitted. 116 116 117 117 For example, the full formulation that uses the complete URN shown at the end of the previous paragraph: ... ... @@ -122,7 +122,7 @@ 122 122 123 123 by omitting all the non-essential parts would become simply: 124 124 125 -> DFR : = DF1 + DF2 119 +> DFR : = DF1 + DF2 126 126 127 127 The references to the Codelists can be simplified similarly. For example, given the non-abbreviated reference to the Codelist AG:CL_FREQ(1.0.0), which is{{footnote}}Single quotes are needed because this reference is not a VTL regular name. 19 Single quotes are not needed in this case because CL_FREQ is a VTL regular name.{{/footnote}}: 128 128 ... ... @@ -168,16 +168,14 @@ 168 168 169 169 === 12.2.5 References to SDMX artefacts from VTL Rulesets === 170 170 171 -The VTL Rulesets allow defining sets of reusable Rules that can be applied by some VTL operators, like the ones for validation and hierarchical roll-up. A "Rule" consists in a relationship between Values belonging to some Value Domains or taken by some Variables, for example: when the Country is USA then the Currency is USD; (ii) the Benelux is composed by Belgium, Luxembourg, Netherlands. 165 +The VTL Rulesets allow defining sets of reusable Rules that can be applied by some VTL operators, like the ones for validation and hierarchical roll-up. A "Rule" consists in a relationship between Values belonging to some Value Domains or taken by some Variables, for example: (i) when the Country is USA then the Currency is USD; (ii) the Benelux is composed by Belgium, Luxembourg, Netherlands. 172 172 173 173 The VTL Rulesets have a signature, in which the Value Domains or the Variables on which the Ruleset is defined are declared, and a body, which contains the Rules. 174 174 175 175 In the signature, given the mapping between VTL and SDMX better described in the following paragraphs, a reference to a VTL Value Domain becomes a reference to a SDMX Codelist, while a reference to a VTL Represented Variable becomes a reference to a SDMX Concept, assuming for it a definite representation{{footnote}}Rulesets of this kind cannot be reused when the referenced Concept has a different representation.{{/footnote}}. 176 176 177 -In general, for referencing SDMX Codelists and Concepts, the conventions described in the previous paragraphs apply. In the Ruleset syntax, the elements that reference SDMX artefacts are called "valueDomain" and "variable" for the Datapoint Rulesets and "ruleValueDomain", "ruleVariable", "condValueDomain" "condVariable" for the Hierarchical Rulesets). 171 +In general, for referencing SDMX Codelists and Concepts, the conventions described in the previous paragraphs apply. In the Ruleset syntax, the elements that reference SDMX artefacts are called "valueDomain" and "variable" for the Datapoint Rulesets and "ruleValueDomain", "ruleVariable", "condValueDomain" "condVariable" for the Hierarchical Rulesets). The syntax of the Ruleset signature allows also to define aliases of the elements above, these aliases are valid only within the specific Ruleset definition statement and cannot be mapped to SDMX.{{footnote}}See also the section "VTL-DL Rulesets" in the VTL Reference Manual.{{/footnote}} 178 178 179 -The syntax of the Ruleset signature allows also to define aliases of the elements above, these aliases are valid only within the specific Ruleset definition statement and cannot be mapped to SDMX.{{footnote}}See also the section "VTL-DL Rulesets" in the VTL Reference Manual.{{/footnote}} 180 - 181 181 In the body of the Rulesets, the Codes and in general all the Values can be written without any other specification, because the artefact, which the Values are referred (Codelist, Concept) to can be deduced from the Ruleset signature. 182 182 183 183 == 12.3 Mapping between SDMX and VTL artefacts == ... ... @@ -460,26 +460,31 @@ 460 460 461 461 > ‘DF2(1.0.0)/GDPPERCAPITA.USA’ <- expression11; ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ <- expression12; 462 462 > … … … 463 -> ‘DF2(1.0.0)/POPGROWTH.USA’ <- expression21; 455 +> ‘DF2(1.0.0)/POPGROWTH.USA’ <- expression21; 464 464 > ‘DF2(1.0.0)/POPGROWTH.CANADA’ <- expression22; 465 465 > … … … 466 466 467 467 As said, it is assumed that these VTL derived Data Sets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 468 468 469 - >VTL dataset INDICATOR value COUNTRY value470 - >471 -> ‘DF2(1.0.0)/GDPPERCAPITA.USA’ GDPPERCAPITA USA 461 +VTL dataset INDICATOR value COUNTRY value 462 + 463 +> ‘DF2(1.0.0)/GDPPERCAPITA.USA’ GDPPERCAPITA USA 472 472 > ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ GDPPERCAPITA CANADA … … … 473 -> 474 -> ‘DF2(1.0.0)/POPGROWTH.USA’ POPGROWTH USA 475 -> ‘DF2(1.0.0)/POPGROWTH.CANADA’ POPGROWTH CANADA 465 +> ‘DF2(1.0.0)/POPGROWTH.USA’ POPGROWTH USA 466 +> ‘DF2(1.0.0)/POPGROWTH.CANADA’ POPGROWTH CANADA 476 476 > … … … 477 477 478 478 It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0.0)), that can be mapped oneto-one to the homonymous SDMX Dataflow. Following the same example, these VTL Transformations would be: 479 479 480 -> DF2bis_GDPPERCAPITA_USA := ‘DF2(1.0.0)/GDPPERCAPITA.USA’ [calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”]; DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”];… … … DF2bis_POPGROWTH_USA := ‘DF2(1.0.0)/POPGROWTH.USA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”USA”]; DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0.0)/POPGROWTH.CANADA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”];… … … DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, DF2bis_GDPPERCAPITA_CANADA’, 471 +> DF2bis_GDPPERCAPITA_USA := ‘DF2(1.0.0)/GDPPERCAPITA.USA’ [calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”]; 472 +> DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”]; … … … 473 +> DF2bis_POPGROWTH_USA := ‘DF2(1.0.0)/POPGROWTH.USA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”USA”]; 474 +> DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0.0)/POPGROWTH.CANADA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”]; … … … 475 +> DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, 476 +> DF2bis_GDPPERCAPITA_CANADA’, 481 481 > … , 482 -> DF2bis_POPGROWTH_USA’, DF2bis_POPGROWTH_CANADA’ 478 +> DF2bis_POPGROWTH_USA’, 479 +> DF2bis_POPGROWTH_CANADA’ 483 483 > …); 484 484 485 485 In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent Data Sets are united and give the final result DF2(1.0){{footnote}}The result is persistent in this example but it can be also non persistent if needed.{{/footnote}}, which can be mapped one-to-one to the homonymous SDMX Dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. ... ... @@ -537,6 +537,7 @@ 537 537 538 538 The VTL data types are sub-divided in scalar types (like integers, strings, etc.), which are the types of the scalar values, and compound types (like Data Sets, Components, Rulesets, etc.), which are the types of the compound structures. See below the diagram of the VTL data types, taken from the VTL User Manual: 539 539 537 + 540 540 [[image:1750070288958-132.png]] 541 541 542 542 **Figure 22 – VTL Data Types** ... ... @@ -572,8 +572,7 @@ 572 572 The following table describes the default mapping for converting from the SDMX data types to the VTL basic scalar types. 573 573 574 574 (% style="width:583.294px" %) 575 -|(% style="width:360px" %)SDMX data type 576 -(BasicComponentDataType)|(% style="width:221px" %)Default VTL basic scalar type 573 +|(% style="width:360px" %)SDMX data type (BasicComponentDataType)|(% style="width:221px" %)Default VTL basic scalar type 577 577 |(% style="width:360px" %)((( 578 578 String 579 579 (string allowing any character) ... ... @@ -608,8 +608,7 @@ 608 608 Short 609 609 (corresponds to XML Schema xs:short datatype; between -32768 and -32767 (inclusive)) 610 610 )))|(% style="width:221px" %)integer 611 -|(% style="width:360px" %)Decimal 612 -(corresponds to XML Schema xs:decimal datatype; subset of real numbers that can be represented as decimals)|(% style="width:221px" %)number 608 +|(% style="width:360px" %)Decimal (corresponds to XML Schema xs:decimal datatype; subset of real numbers that can be represented as decimals)|(% style="width:221px" %)number 613 613 |(% style="width:360px" %)((( 614 614 Float 615 615 (corresponds to XML Schema xs:float datatype; patterned after the IEEE single-precision 32-bit floating point type) ... ... @@ -659,7 +659,7 @@ 659 659 GregorianTimePeriod 660 660 (superset of GregorianYear, GregorianYearMonth, and GregorianDay) 661 661 )))|(% style="width:221px" %)date 662 -|(% style="width:360px" %)GregorianYear (YYYY)|(% style="width:221px" %)date 658 +|(% style="width:360px" %)GregorianYear (YYYY)|(% style="width:221px" %)date 663 663 |(% style="width:360px" %)GregorianYearMonth / GregorianMonth (YYYY-MM)|(% style="width:221px" %)date 664 664 |(% style="width:360px" %)GregorianDay (YYYY-MM-DD)|(% style="width:221px" %)date 665 665 |(% style="width:360px" %)((( ... ... @@ -698,19 +698,19 @@ 698 698 )))|(% style="width:221px" %)date 699 699 |(% style="width:360px" %)((( 700 700 TimeRange 701 -(YYYY-MM-DD(Thh:mm:ss)?/) 697 +(YYYY-MM-DD(Thh:mm:ss)?/<duration>) 702 702 )))|(% style="width:221px" %)time 703 703 |(% style="width:360px" %)((( 704 704 Month 705 -(MM; speicifies a month independent of a year; e.g. February is black history month in the United States) 701 +(~-~-MM; speicifies a month independent of a year; e.g. February is black history month in the United States) 706 706 )))|(% style="width:221px" %)string 707 707 |(% style="width:360px" %)((( 708 708 MonthDay 709 -(MM-DD; specifies a day within a month independent of a year; e.g. Christmas is December 25^^th^^; used to specify reporting year start day) 705 +(~-~-MM-DD; specifies a day within a month independent of a year; e.g. Christmas is December 25^^th^^; used to specify reporting year start day) 710 710 )))|(% style="width:221px" %)string 711 711 |(% style="width:360px" %)((( 712 712 Day 713 -(-DD; specifies a day independent of a month or year; e.g. the 15^^th^^ is payday) 709 +(~-~--DD; specifies a day independent of a month or year; e.g. the 15^^th^^ is payday) 714 714 )))|(% style="width:221px" %)string 715 715 |(% style="width:360px" %)((( 716 716 Time
- SKMS.Methodology.Code.MethodologyClass[0]
-
- Index
-
... ... @@ -1,1 +1,0 @@ 1 -12
- SUZ.Methodology.Code.MethodologyClass[0]
-