Changes for page 12 Validation and Transformation Language (VTL)
Last modified by Helena on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 8 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -414,13 +414,8 @@ 414 414 415 415 It should be noted that the desired VTL Data Sets (i.e. of the kind ‘DF1(1.0.0)/// INDICATORvalue//.//COUNTRYvalue//’) can be obtained also by applying the VTL operator “**sub**” (subspace) to the Dataflow DF1(1.0.0), like in the following VTL expression: 416 416 417 -‘DF1(1.0.0)/POPULATION.USA’ := 418 -DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA” ]; 417 +[[image:1747388275998-621.png]] 419 419 420 -‘DF1(1.0.0)/POPULATION.CANADA’ := 421 -DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“CANADA” ]; 422 -… … … 423 - 424 424 In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX Dataflow to different VTL Data Sets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a Dataflow.{{footnote}}In case the ordered concatenation notation is used, the VTL Transformation described above, e.g. ‘DF1(1.0)/POPULATION.USA’ := DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA”], is implicitly executed. In order to test the overall compliance of the VTL program to the VTL consistency rules, it has to be considered as part of the VTL program even if it is not explicitly coded.{{/footnote}} 425 425 426 426 In the direction from SDMX to VTL it is allowed to omit the value of one or more ... ... @@ -431,8 +431,7 @@ 431 431 432 432 This is equivalent to the application of the VTL “sub” operator only to the identifier //INDICATOR//: 433 433 434 -‘DF1(1.0.0)/POPULATION.’ := 435 -DF1(1.0.0) [ sub INDICATOR=“POPULATION” ]; 429 +[[image:1747388244829-693.png]] 436 436 437 437 Therefore the VTL Data Set ‘DF1(1.0.0)/POPULATION.’ would have the identifiers COUNTRY and TIME_PERIOD. 438 438 ... ... @@ -459,36 +459,17 @@ 459 459 460 460 Some examples follow, for some specific values of INDICATOR and COUNTRY: 461 461 462 - ‘DF2(1.0.0)/GDPPERCAPITA.USA’ <- expression11; ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ <- expression12;456 +[[image:1747388222879-916.png]] 463 463 464 -… … … 465 -‘DF2(1.0.0)/POPGROWTH.USA’ <- expression21; 466 -‘DF2(1.0.0)/POPGROWTH.CANADA’ <- expression22; 467 -… … … 458 +[[image:1747388206717-256.png]] 468 468 469 469 As said, it is assumed that these VTL derived Data Sets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 470 470 471 - VTL dataset INDICATOR value COUNTRY value462 +[[image:1747388148322-387.png]] 472 472 473 -‘DF2(1.0.0)/GDPPERCAPITA.USA’ GDPPERCAPITA USA 474 -‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ GDPPERCAPITA CANADA … … … 475 -‘DF2(1.0.0)/POPGROWTH.USA’ POPGROWTH USA 476 -‘DF2(1.0.0)/POPGROWTH.CANADA’ POPGROWTH CANADA 477 -… … … 478 - 479 479 It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0.0)), that can be mapped oneto-one to the homonymous SDMX Dataflow. Following the same example, these VTL Transformations would be: 480 480 481 -DF2bis_GDPPERCAPITA_USA := ‘DF2(1.0.0)/GDPPERCAPITA.USA’ [calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”]; 482 -DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”]; … … … 483 -DF2bis_POPGROWTH_USA := ‘DF2(1.0.0)/POPGROWTH.USA’ 484 -[calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”USA”]; 485 -DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0.0)/POPGROWTH.CANADA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”]; … … … 486 -DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, 487 -DF2bis_GDPPERCAPITA_CANADA’, 488 -… , 489 -DF2bis_POPGROWTH_USA’, 490 -DF2bis_POPGROWTH_CANADA’ 491 -…); 466 +[[image:1747388179021-814.png]] 492 492 493 493 In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example 494 494 ... ... @@ -502,38 +502,30 @@ 502 502 503 503 With reference to the VTL “model for Variables and Value domains”, the following additional mappings have to be considered: 504 504 505 -(% style="width: 1170.29px" %)506 -|**VTL**|(% style="width: 754px" %)**SDMX**507 -|**Data Set Component**|(% style="width: 754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}}508 -|**Represented Variable**|(% style="width: 754px" %)(((480 +(% style="width:895.294px" %) 481 +|(% style="width:278px" %)**VTL**|(% style="width:613px" %)**SDMX** 482 +|(% style="width:278px" %)**Data Set Component**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}} 483 +|(% style="width:278px" %)**Represented Variable**|(% style="width:613px" %)((( 509 509 **Concept** with a definite 510 510 511 511 Representation 512 512 ))) 513 -|**Value Domain**|(% style="width:754px" %)((( 514 -**Representation** (see the Structure 515 - 516 -Pattern in the Base Package) 488 +|(% style="width:278px" %)**Value Domain**|(% style="width:613px" %)((( 489 +**Representation** (see the Structure Pattern in the Base Package) 517 517 ))) 518 -|**Enumerated Value Domain / Code List**|(% style="width:754px" %)**Codelist** 519 -|**Code**|(% style="width:754px" %)((( 520 -**Code** (for enumerated 521 - 522 -DimensionComponent, Measure, DataAttribute) 491 +|(% style="width:278px" %)**Enumerated Value Domain / 492 +Code List**|(% style="width:613px" %)**Codelist** 493 +|(% style="width:278px" %)**Code**|(% style="width:613px" %)((( 494 +**Code** (for enumerated DimensionComponent, Measure, DataAttribute) 523 523 ))) 524 -|**Described Value Domain**|(% style="width:754px" %)((( 525 -non-enumerated** Representation** 526 - 527 -(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 496 +|(% style="width:278px" %)**Described Value Domain**|(% style="width:613px" %)((( 497 +non-enumerated** Representation **(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 528 528 ))) 529 -|**Value**|(% style="width:754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or 530 -| |(% style="width:754px" %)((( 531 -to a valid **value **(for non-enumerated** **Representations) 532 -))) 533 -|**Value Domain Subset / Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 534 -|**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 535 -|**Described Value Domain Subset / Described Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 536 -|**Set list**|(% style="width:754px" %)This abstraction does not exist in SDMX 499 +|(% style="width:278px" %)**Value**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or to a valid **value **(for non-enumerated** **Representations) 500 +|(% style="width:278px" %)**Value Domain Subset / Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 501 +|(% style="width:278px" %)**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 502 +|(% style="width:278px" %)**Described Value Domain Subset / Described Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 503 +|(% style="width:278px" %)**Set list**|(% style="width:613px" %)This abstraction does not exist in SDMX 537 537 538 538 The main difference between VTL and SDMX relies on the fact that the VTL artefacts for defining subsets of Value Domains do not exist in SDMX, therefore the VTL features for referring to predefined subsets are not available in SDMX. These artefacts are the Value Domain Subset (or Set), either enumerated or described, the Set List (list of values belonging to enumerated subsets) and the Data Set Component (aimed at defining the set of values that the Component of a Data Set can take, possibly a subset of the codes of Value Domain). 539 539 ... ... @@ -559,7 +559,7 @@ 559 559 560 560 The VTL data types are sub-divided in scalar types (like integers, strings, etc.), which are the types of the scalar values, and compound types (like Data Sets, Components, Rulesets, etc.), which are the types of the compound structures. See below the diagram of the VTL data types, taken from the VTL User Manual: 561 561 562 -[[image: SDMX3-0-0 SECTION6FINAL-1.0_en_e3df33ae.png||height="543" width="483"]]529 +[[image:1747388434672-948.png]] 563 563 564 564 (% class="wikigeneratedid" id="HFigure222013VTLDataTypes" %) 565 565 **Figure 22 – VTL Data Types** ... ... @@ -568,13 +568,10 @@ 568 568 569 569 The VTL basic scalar types are listed below and follow a hierarchical structure in terms of supersets/subsets (e.g. "scalar" is the superset of all the basic scalar types): 570 570 538 +[[image:1747388465321-274.png]] 571 571 572 572 **Figure 23 – VTL Basic Scalar Types** 573 573 574 -((( 575 - 576 -))) 577 - 578 578 === 12.4.2 VTL basic scalar types and SDMX data types === 579 579 580 580 The VTL assumes that a basic scalar type has a unique internal representation and can have more external representations. ... ... @@ -788,53 +788,53 @@ 788 788 789 789 The custom output formats can be specified by means of the VTL formatting mask described in the section "Type Conversion and Formatting Mask" of the VTL Reference Manual. Such a section describes the masks for the VTL basic scalar types "number", "integer", "date", "time", "time_period" and "duration" and gives examples. As for the types "string" and "boolean" the VTL conventions are extended with some other special characters as described in the following table. 790 790 791 - |(%colspan="2" %)VTL special characters for the formatting masks792 -|(% colspan="2" %) 793 -|(% colspan="2" %) Number794 -| D|one numeric digit(ifthe scientific notationisadopted, Dis only forthemantissa)795 -| E|one numeric digit (fortheexponent of thescientific notation)796 -| .(dot)|possibleseparatorbetweentheintegerandthedecimalparts.797 -| ,(comma)|possible separator between the integer and the decimal parts.798 -| | 799 -|(% colspan="2" %)Timeand duration800 -| C|century801 -| Y|year802 -| S|semester803 -| Q|quarter804 -| M|month805 -| W|week806 -| D|day807 -| h|hourdigit(bydefaulton 24hours)808 -| M|minute809 -| S|second810 -| D|decimalofsecond811 -| P|periodindicator (representationin onedigitfortheduration)812 -|P| numberofthe periodsspecifiedntheperiodindicator813 -| AM/PM|indicator ofAM/PM(e.g.am/pm for"am"or"pm")814 -|M ONTH|uppercasetextualrepresentationofthemonth(e.g.,JANUARYforJanuary)815 -| DAY|uppercase textual representation of theday(e.g.,MONDAY forMonday)816 -| Month|lowercase textual representation of themonth(e.g.,january)817 -| Day|lowercase textual representation of the month (e.g.,monday)818 -| Month|Firstcharacter uppercase,thenlowercase textual representation of the month (e.g.,January)819 -| Day|First character uppercase, then lowercase textual representation of theday using(e.g.Monday)820 -| 821 -|(% colspan="2" %)String822 -| X|anystring character823 -| Z|any string characterfrom "A" to "z"824 -| 9|any string character from "0" to "9"825 -| 826 -|(% colspan="2" %)Boolean827 -| B|Booleanusing "true"for True andfalse"for False828 -| 1|Boolean using "1" for True and "0" for False829 -| 0|Boolean using "0" for True and "1" for False830 -| 831 -|(% colspan="2" %)Otherqualifiers832 -| *|anarbitrarynumber of digits(oftheprecedingtype)833 -| +|atleastone digit (of the preceding type)834 -|( )|optional digits(specifiedwithin thebrackets)835 -| \|prefix forthespecialcharactersthat must appearin themask836 -| N|fixednumber of digitsusedinthe preceding textual representationof themonthor theday837 -| 755 +(% style="width:713.294px" %) 756 +|(% colspan="2" style="width:710px" %)VTL special characters for the formatting masks 757 +|(% colspan="2" style="width:710px" %) 758 +|(% colspan="2" style="width:710px" %)Number 759 +|D|(% style="width:486px" %)one numeric digit (if the scientific notation is adopted, D is only for the mantissa) 760 +|E|(% style="width:486px" %)one numeric digit (for the exponent of the scientific notation) 761 +|. (dot)|(% style="width:486px" %)possible separator between the integer and the decimal parts. 762 +|, (comma)|(% style="width:486px" %)possible separator between the integer and the decimal parts. 763 +| |(% style="width:486px" %) 764 +|(% colspan="2" style="width:710px" %)Time and duration 765 +|C|(% style="width:486px" %)century 766 +|Y|(% style="width:486px" %)year 767 +|S|(% style="width:486px" %)semester 768 +|Q|(% style="width:486px" %)quarter 769 +|M|(% style="width:486px" %)month 770 +|W|(% style="width:486px" %)week 771 +|D|(% style="width:486px" %)day 772 +|h|(% style="width:486px" %)hour digit (by default on 24 hours) 773 +|M|(% style="width:486px" %)minute 774 +|S|(% style="width:486px" %)second 775 +|D|(% style="width:486px" %)decimal of second 776 +|P|(% style="width:486px" %)period indicator (representation in one digit for the duration) 777 +|P|(% style="width:486px" %)number of the periods specified in the period indicator 778 +|AM/PM|(% style="width:486px" %)indicator of AM / PM (e.g. am/pm for "am" or "pm") 779 +|MONTH|(% style="width:486px" %)uppercase textual representation of the month (e.g., JANUARY for January) 780 +|DAY|(% style="width:486px" %)uppercase textual representation of the day (e.g., MONDAY for Monday) 781 +|Month|(% style="width:486px" %)lowercase textual representation of the month (e.g., january) 782 +|Day|(% style="width:486px" %)lowercase textual representation of the month (e.g., monday) 783 +|Month|(% style="width:486px" %)First character uppercase, then lowercase textual representation of the month (e.g., January) 784 +|Day|(% style="width:486px" %)First character uppercase, then lowercase textual representation of the day using (e.g. Monday) 785 +| |(% style="width:486px" %) 786 +|(% colspan="2" style="width:710px" %)String 787 +|X|(% style="width:486px" %)any string character 788 +|Z|(% style="width:486px" %)any string character from "A" to "z" 789 +|9|(% style="width:486px" %)any string character from "0" to "9" 790 +| |(% style="width:486px" %) 791 +|(% colspan="2" style="width:710px" %)Boolean 792 +|B|(% style="width:486px" %)Boolean using "true" for True and "false" for False 793 +|1|(% style="width:486px" %)Boolean using "1" for True and "0" for False 794 +|0|(% style="width:486px" %)Boolean using "0" for True and "1" for False 795 +| |(% style="width:486px" %) 796 +|(% colspan="2" style="width:710px" %)Other qualifiers 797 +|*|(% style="width:486px" %)an arbitrary number of digits (of the preceding type) 798 +|+|(% style="width:486px" %)at least one digit (of the preceding type) 799 +|( )|(% style="width:486px" %)optional digits (specified within the brackets) 800 +|\|(% style="width:486px" %)prefix for the special characters that must appear in the mask 801 +|N|(% style="width:486px" %)fixed number of digits used in the preceding textual representation of the month or the day 838 838 839 839 The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL Transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion{{footnote}}The representation given in the DSD should obviously be compatible with the VTL data type.{{/footnote}}. 840 840
- 1747388148322-387.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +23.0 KB - Content
- 1747388179021-814.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +66.2 KB - Content
- 1747388206717-256.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +10.3 KB - Content
- 1747388222879-916.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +9.9 KB - Content
- 1747388244829-693.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +7.4 KB - Content
- 1747388275998-621.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +19.2 KB - Content
- 1747388434672-948.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +43.2 KB - Content
- 1747388465321-274.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +17.9 KB - Content