Changes for page 12 Validation and Transformation Language (VTL)
Last modified by Helena on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 0 added, 8 removed)
Details
- Page properties
-
- Content
-
... ... @@ -414,8 +414,13 @@ 414 414 415 415 It should be noted that the desired VTL Data Sets (i.e. of the kind ‘DF1(1.0.0)/// INDICATORvalue//.//COUNTRYvalue//’) can be obtained also by applying the VTL operator “**sub**” (subspace) to the Dataflow DF1(1.0.0), like in the following VTL expression: 416 416 417 -[[image:1747388275998-621.png]] 417 +‘DF1(1.0.0)/POPULATION.USA’ := 418 +DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA” ]; 418 418 420 +‘DF1(1.0.0)/POPULATION.CANADA’ := 421 +DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“CANADA” ]; 422 +… … … 423 + 419 419 In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX Dataflow to different VTL Data Sets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a Dataflow.{{footnote}}In case the ordered concatenation notation is used, the VTL Transformation described above, e.g. ‘DF1(1.0)/POPULATION.USA’ := DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA”], is implicitly executed. In order to test the overall compliance of the VTL program to the VTL consistency rules, it has to be considered as part of the VTL program even if it is not explicitly coded.{{/footnote}} 420 420 421 421 In the direction from SDMX to VTL it is allowed to omit the value of one or more ... ... @@ -426,7 +426,8 @@ 426 426 427 427 This is equivalent to the application of the VTL “sub” operator only to the identifier //INDICATOR//: 428 428 429 -[[image:1747388244829-693.png]] 434 +‘DF1(1.0.0)/POPULATION.’ := 435 +DF1(1.0.0) [ sub INDICATOR=“POPULATION” ]; 430 430 431 431 Therefore the VTL Data Set ‘DF1(1.0.0)/POPULATION.’ would have the identifiers COUNTRY and TIME_PERIOD. 432 432 ... ... @@ -453,18 +453,47 @@ 453 453 454 454 Some examples follow, for some specific values of INDICATOR and COUNTRY: 455 455 456 - [[image:1747388222879-916.png]]462 +‘DF2(1.0.0)/GDPPERCAPITA.USA’ <- expression11; ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ <- expression12; 457 457 458 -[[image:1747388206717-256.png]] 464 +… … … 465 +‘DF2(1.0.0)/POPGROWTH.USA’ <- expression21; 466 +‘DF2(1.0.0)/POPGROWTH.CANADA’ <- expression22; 467 +… … … 459 459 460 460 As said, it is assumed that these VTL derived Data Sets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 461 461 462 - [[image:1747388148322-387.png]]471 +VTL dataset INDICATOR value COUNTRY value 463 463 473 +‘DF2(1.0.0)/GDPPERCAPITA.USA’ GDPPERCAPITA USA 474 +‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ GDPPERCAPITA CANADA … … … 475 +‘DF2(1.0.0)/POPGROWTH.USA’ POPGROWTH USA 476 +‘DF2(1.0.0)/POPGROWTH.CANADA’ POPGROWTH CANADA 477 +… … … 478 + 464 464 It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0.0)), that can be mapped oneto-one to the homonymous SDMX Dataflow. Following the same example, these VTL Transformations would be: 465 465 466 - [[image:1747388179021-814.png]]481 +DF2bis_GDPPERCAPITA_USA := ‘DF2(1.0.0)/GDPPERCAPITA.USA’ [calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”]; 467 467 483 +DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”]; … … … 484 + 485 +DF2bis_POPGROWTH_USA := ‘DF2(1.0.0)/POPGROWTH.USA’ 486 + 487 +[calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”USA”]; 488 + 489 +DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0.0)/POPGROWTH.CANADA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”]; … … … 490 + 491 +DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, 492 + 493 +DF2bis_GDPPERCAPITA_CANADA’, 494 + 495 +… , 496 + 497 +DF2bis_POPGROWTH_USA’, 498 + 499 +DF2bis_POPGROWTH_CANADA’ 500 + 501 +…); 502 + 468 468 In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example 469 469 470 470 DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent Data Sets are united and give the final result DF2(1.0){{footnote}}The result is persistent in this example but it can be also non persistent if needed.{{/footnote}}, which can be mapped one-to-one to the homonymous SDMX Dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. ... ... @@ -477,30 +477,38 @@ 477 477 478 478 With reference to the VTL “model for Variables and Value domains”, the following additional mappings have to be considered: 479 479 480 -(% style="width: 895.294px" %)481 -| (% style="width:278px" %)**VTL**|(% style="width:613px" %)**SDMX**482 -| (% style="width:278px" %)**Data Set Component**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}}483 -| (% style="width:278px" %)**Represented Variable**|(% style="width:613px" %)(((515 +(% style="width:1170.29px" %) 516 +|**VTL**|(% style="width:754px" %)**SDMX** 517 +|**Data Set Component**|(% style="width:754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}} 518 +|**Represented Variable**|(% style="width:754px" %)((( 484 484 **Concept** with a definite 485 485 486 486 Representation 487 487 ))) 488 -|(% style="width:278px" %)**Value Domain**|(% style="width:613px" %)((( 489 -**Representation** (see the Structure Pattern in the Base Package) 523 +|**Value Domain**|(% style="width:754px" %)((( 524 +**Representation** (see the Structure 525 + 526 +Pattern in the Base Package) 490 490 ))) 491 -|(% style="width:278px" %)**Enumerated Value Domain / 492 -Code List**|(% style="width:613px" %)**Codelist** 493 -|(% style="width:278px" %)**Code**|(% style="width:613px" %)((( 494 -**Code** (for enumerated DimensionComponent, Measure, DataAttribute) 528 +|**Enumerated Value Domain / Code List**|(% style="width:754px" %)**Codelist** 529 +|**Code**|(% style="width:754px" %)((( 530 +**Code** (for enumerated 531 + 532 +DimensionComponent, Measure, DataAttribute) 495 495 ))) 496 -|(% style="width:278px" %)**Described Value Domain**|(% style="width:613px" %)((( 497 -non-enumerated** Representation **(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 534 +|**Described Value Domain**|(% style="width:754px" %)((( 535 +non-enumerated** Representation** 536 + 537 +(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 498 498 ))) 499 -|(% style="width:278px" %)**Value**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or to a valid **value **(for non-enumerated** **Representations) 500 -|(% style="width:278px" %)**Value Domain Subset / Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 501 -|(% style="width:278px" %)**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 502 -|(% style="width:278px" %)**Described Value Domain Subset / Described Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 503 -|(% style="width:278px" %)**Set list**|(% style="width:613px" %)This abstraction does not exist in SDMX 539 +|**Value**|(% style="width:754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or 540 +| |(% style="width:754px" %)((( 541 +to a valid **value **(for non-enumerated** **Representations) 542 +))) 543 +|**Value Domain Subset / Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 544 +|**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 545 +|**Described Value Domain Subset / Described Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 546 +|**Set list**|(% style="width:754px" %)This abstraction does not exist in SDMX 504 504 505 505 The main difference between VTL and SDMX relies on the fact that the VTL artefacts for defining subsets of Value Domains do not exist in SDMX, therefore the VTL features for referring to predefined subsets are not available in SDMX. These artefacts are the Value Domain Subset (or Set), either enumerated or described, the Set List (list of values belonging to enumerated subsets) and the Data Set Component (aimed at defining the set of values that the Component of a Data Set can take, possibly a subset of the codes of Value Domain). 506 506 ... ... @@ -526,7 +526,7 @@ 526 526 527 527 The VTL data types are sub-divided in scalar types (like integers, strings, etc.), which are the types of the scalar values, and compound types (like Data Sets, Components, Rulesets, etc.), which are the types of the compound structures. See below the diagram of the VTL data types, taken from the VTL User Manual: 528 528 529 -[[image: 1747388434672-948.png]]572 +[[image:SDMX 3-0-0 SECTION 6 FINAL-1.0_en_e3df33ae.png||height="543" width="483"]] 530 530 531 531 (% class="wikigeneratedid" id="HFigure222013VTLDataTypes" %) 532 532 **Figure 22 – VTL Data Types** ... ... @@ -535,10 +535,13 @@ 535 535 536 536 The VTL basic scalar types are listed below and follow a hierarchical structure in terms of supersets/subsets (e.g. "scalar" is the superset of all the basic scalar types): 537 537 538 -[[image:1747388465321-274.png]] 539 539 540 540 **Figure 23 – VTL Basic Scalar Types** 541 541 584 +((( 585 + 586 +))) 587 + 542 542 === 12.4.2 VTL basic scalar types and SDMX data types === 543 543 544 544 The VTL assumes that a basic scalar type has a unique internal representation and can have more external representations. ... ... @@ -752,53 +752,53 @@ 752 752 753 753 The custom output formats can be specified by means of the VTL formatting mask described in the section "Type Conversion and Formatting Mask" of the VTL Reference Manual. Such a section describes the masks for the VTL basic scalar types "number", "integer", "date", "time", "time_period" and "duration" and gives examples. As for the types "string" and "boolean" the VTL conventions are extended with some other special characters as described in the following table. 754 754 755 -(% style="width:713.294px" %)756 -|(% colspan="2" style="width:710px"%)VTLspecial characters for the formatting masks757 -|(% colspan="2" style="width:710px"%)758 -|( %colspan="2"style="width:710px"%)Number759 -| D|(% style="width:486px" %)one numeric digit (if the scientific notationis adopted, D is only for the mantissa)760 -| E|(%style="width:486px" %)onenumeric digit(for thexponentofthescientificnotation)761 -| .(dot)|(% style="width:486px" %)possible separator between the integer and the decimal parts.762 -| ,(comma)|(%style="width:486px" %)possible separator between the integer and the decimal parts.763 -| |(%style="width:486px" %)764 -| (%colspan="2" style="width:710px" %)Time and duration765 -| C|(% style="width:486px" %)century766 -| Y|(%style="width:486px" %)year767 -| S|(% style="width:486px" %)semester768 -| Q|(% style="width:486px" %)quarter769 -| M|(% style="width:486px" %)month770 -| W|(% style="width:486px" %)week771 -| D|(%style="width:486px"%)day772 -| h|(% style="width:486px" %)hour digit(by default on 24 hours)773 -| M|(%style="width:486px" %)minute774 -| S|(% style="width:486px"%)second775 -| D|(% style="width:486px"%)decimalofsecond776 -|P| (%style="width:486px"%)period indicator(representationinonedigit for the duration)777 -|P| (% style="width:486px" %)number oftheperiodsspecifiedin the periodindicator778 -| AM/PM|(%style="width:486px"%)indicatorofAM/PM(e.g.am/pmfor"am" or"pm")779 -| MONTH|(% style="width:486px" %)uppercase textual representation of themonth(e.g.,JANUARY forJanuary)780 -| DAY|(% style="width:486px" %)uppercase textual representation of theday(e.g.,MONDAY for Monday)781 -| Month|(% style="width:486px" %)lowercase textual representation of the month (e.g.,january)782 -| Day|(%style="width:486px"%)lowercase textual representation of the month (e.g.,monday)783 -| Month|(% style="width:486px" %)First character uppercase, then lowercase textual representation of themonth(e.g.,January)784 -| Day|(%style="width:486px" %)First character uppercase, then lowercase textual representation of the day using (e.g. Monday)785 -| |(%style="width:486px" %)786 -| (% colspan="2"style="width:710px" %)String787 -| X|(% style="width:486px" %)any string character788 -| Z|(% style="width:486px" %)any string character from "A" to "z"789 -| 9|(%style="width:486px" %)any string character from "0" to "9"790 -| |(%style="width:486px" %)791 -| (% colspan="2"style="width:710px"%)Boolean792 -| B|(% style="width:486px" %)Boolean using "true" for True and "false" for False793 -| 1|(% style="width:486px" %)Boolean using "1" for True and "0" for False794 -| 0|(%style="width:486px" %)Boolean using "0" for True and "1" for False795 -| |(%style="width:486px" %)796 -| (% colspan="2"style="width:710px"%)Otherqualifiers797 -| *|(% style="width:486px" %)an arbitrarynumberofdigits(of the preceding type)798 -| +|(%style="width:486px" %)atleastonedigit (of theprecedingtype)799 -| ( )|(% style="width:486px"%)optionaldigits(specifiedwithin thebrackets)800 -| \|(% style="width:486px"%)prefixforthespecialcharactershatmustappearin themask801 -| N|(%style="width:486px" %)fixed number of digits used in the preceding textual representation of the month or the day801 +|(% colspan="2" %)VTL special characters for the formatting masks 802 +|(% colspan="2" %) 803 +|(% colspan="2" %)Number 804 +|D|one numeric digit (if the scientific notation is adopted, D is only for the mantissa) 805 +|E|one numeric digit (for the exponent of the scientific notation) 806 +|. (dot)|possible separator between the integer and the decimal parts. 807 +|, (comma)|possible separator between the integer and the decimal parts. 808 +| | 809 +|(% colspan="2" %)Time and duration 810 +|C|century 811 +|Y|year 812 +|S|semester 813 +|Q|quarter 814 +|M|month 815 +|W|week 816 +|D|day 817 +|h|hour digit (by default on 24 hours) 818 +|M|minute 819 +|S|second 820 +|D|decimal of second 821 +|P|period indicator (representation in one digit for the duration) 822 +|P|number of the periods specified in the period indicator 823 +|AM/PM|indicator of AM / PM (e.g. am/pm for "am" or "pm") 824 +|MONTH|uppercase textual representation of the month (e.g., JANUARY for January) 825 +|DAY|uppercase textual representation of the day (e.g., MONDAY for Monday) 826 +|Month|lowercase textual representation of the month (e.g., january) 827 +|Day|lowercase textual representation of the month (e.g., monday) 828 +|Month|First character uppercase, then lowercase textual representation of the month (e.g., January) 829 +|Day|First character uppercase, then lowercase textual representation of the day using (e.g. Monday) 830 +| | 831 +|(% colspan="2" %)String 832 +|X|any string character 833 +|Z|any string character from "A" to "z" 834 +|9|any string character from "0" to "9" 835 +| | 836 +|(% colspan="2" %)Boolean 837 +|B|Boolean using "true" for True and "false" for False 838 +|1|Boolean using "1" for True and "0" for False 839 +|0|Boolean using "0" for True and "1" for False 840 +| | 841 +|(% colspan="2" %)Other qualifiers 842 +|*|an arbitrary number of digits (of the preceding type) 843 +|+|at least one digit (of the preceding type) 844 +|( )|optional digits (specified within the brackets) 845 +|\|prefix for the special characters that must appear in the mask 846 +|N|fixed number of digits used in the preceding textual representation of the month or the day 847 +| | 802 802 803 803 The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL Transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion{{footnote}}The representation given in the DSD should obviously be compatible with the VTL data type.{{/footnote}}. 804 804
- 1747388148322-387.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -23.0 KB - Content
- 1747388179021-814.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -66.2 KB - Content
- 1747388206717-256.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -10.3 KB - Content
- 1747388222879-916.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -9.9 KB - Content
- 1747388244829-693.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -7.4 KB - Content
- 1747388275998-621.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -19.2 KB - Content
- 1747388434672-948.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -43.2 KB - Content
- 1747388465321-274.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -17.9 KB - Content