Changes for page 12 Validation and Transformation Language (VTL)
Last modified by Helena on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 8 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -414,14 +414,8 @@ 414 414 415 415 It should be noted that the desired VTL Data Sets (i.e. of the kind ‘DF1(1.0.0)/// INDICATORvalue//.//COUNTRYvalue//’) can be obtained also by applying the VTL operator “**sub**” (subspace) to the Dataflow DF1(1.0.0), like in the following VTL expression: 416 416 417 -‘DF1(1.0.0)/POPULATION.USA’ := 418 -DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA” ]; 417 +[[image:1747388275998-621.png]] 419 419 420 -‘DF1(1.0.0)/POPULATION.CANADA’ := 421 -DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“CANADA” ]; 422 - 423 -… … … 424 - 425 425 In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX Dataflow to different VTL Data Sets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a Dataflow.{{footnote}}In case the ordered concatenation notation is used, the VTL Transformation described above, e.g. ‘DF1(1.0)/POPULATION.USA’ := DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA”], is implicitly executed. In order to test the overall compliance of the VTL program to the VTL consistency rules, it has to be considered as part of the VTL program even if it is not explicitly coded.{{/footnote}} 426 426 427 427 In the direction from SDMX to VTL it is allowed to omit the value of one or more ... ... @@ -432,8 +432,7 @@ 432 432 433 433 This is equivalent to the application of the VTL “sub” operator only to the identifier //INDICATOR//: 434 434 435 -‘DF1(1.0.0)/POPULATION.’ := 436 -DF1(1.0.0) [ sub INDICATOR=“POPULATION” ]; 429 +[[image:1747388244829-693.png]] 437 437 438 438 Therefore the VTL Data Set ‘DF1(1.0.0)/POPULATION.’ would have the identifiers COUNTRY and TIME_PERIOD. 439 439 ... ... @@ -460,51 +460,18 @@ 460 460 461 461 Some examples follow, for some specific values of INDICATOR and COUNTRY: 462 462 463 - ‘DF2(1.0.0)/GDPPERCAPITA.USA’ <- expression11; ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ <- expression12;456 +[[image:1747388222879-916.png]] 464 464 465 -… … … 466 -‘DF2(1.0.0)/POPGROWTH.USA’ <- expression21; 467 -‘DF2(1.0.0)/POPGROWTH.CANADA’ <- expression22; 468 -… … … 458 +[[image:1747388206717-256.png]] 469 469 470 470 As said, it is assumed that these VTL derived Data Sets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 471 471 472 - VTL dataset INDICATOR value COUNTRY value462 +[[image:1747388148322-387.png]] 473 473 474 -‘DF2(1.0.0)/GDPPERCAPITA.USA’ GDPPERCAPITA USA 475 - 476 -‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ GDPPERCAPITA CANADA … … … 477 - 478 -‘DF2(1.0.0)/POPGROWTH.USA’ POPGROWTH USA 479 - 480 -‘DF2(1.0.0)/POPGROWTH.CANADA’ POPGROWTH CANADA 481 - 482 -… … … 483 - 484 484 It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0.0)), that can be mapped oneto-one to the homonymous SDMX Dataflow. Following the same example, these VTL Transformations would be: 485 485 486 - DF2bis_GDPPERCAPITA_USA:= ‘DF2(1.0.0)/GDPPERCAPITA.USA’ [calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”];466 +[[image:1747388179021-814.png]] 487 487 488 -DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”]; … … … 489 - 490 -DF2bis_POPGROWTH_USA := ‘DF2(1.0.0)/POPGROWTH.USA’ 491 - 492 -[calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”USA”]; 493 - 494 -DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0.0)/POPGROWTH.CANADA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”]; … … … 495 - 496 -DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, 497 - 498 -DF2bis_GDPPERCAPITA_CANADA’, 499 - 500 -… , 501 - 502 -DF2bis_POPGROWTH_USA’, 503 - 504 -DF2bis_POPGROWTH_CANADA’ 505 - 506 -…); 507 - 508 508 In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example 509 509 510 510 DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent Data Sets are united and give the final result DF2(1.0){{footnote}}The result is persistent in this example but it can be also non persistent if needed.{{/footnote}}, which can be mapped one-to-one to the homonymous SDMX Dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. ... ... @@ -517,38 +517,30 @@ 517 517 518 518 With reference to the VTL “model for Variables and Value domains”, the following additional mappings have to be considered: 519 519 520 -(% style="width: 1170.29px" %)521 -|**VTL**|(% style="width: 754px" %)**SDMX**522 -|**Data Set Component**|(% style="width: 754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}}523 -|**Represented Variable**|(% style="width: 754px" %)(((480 +(% style="width:895.294px" %) 481 +|(% style="width:278px" %)**VTL**|(% style="width:613px" %)**SDMX** 482 +|(% style="width:278px" %)**Data Set Component**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}} 483 +|(% style="width:278px" %)**Represented Variable**|(% style="width:613px" %)((( 524 524 **Concept** with a definite 525 525 526 526 Representation 527 527 ))) 528 -|**Value Domain**|(% style="width:754px" %)((( 529 -**Representation** (see the Structure 530 - 531 -Pattern in the Base Package) 488 +|(% style="width:278px" %)**Value Domain**|(% style="width:613px" %)((( 489 +**Representation** (see the Structure Pattern in the Base Package) 532 532 ))) 533 -|**Enumerated Value Domain / Code List**|(% style="width:754px" %)**Codelist** 534 -|**Code**|(% style="width:754px" %)((( 535 -**Code** (for enumerated 536 - 537 -DimensionComponent, Measure, DataAttribute) 491 +|(% style="width:278px" %)**Enumerated Value Domain / 492 +Code List**|(% style="width:613px" %)**Codelist** 493 +|(% style="width:278px" %)**Code**|(% style="width:613px" %)((( 494 +**Code** (for enumerated DimensionComponent, Measure, DataAttribute) 538 538 ))) 539 -|**Described Value Domain**|(% style="width:754px" %)((( 540 -non-enumerated** Representation** 541 - 542 -(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 496 +|(% style="width:278px" %)**Described Value Domain**|(% style="width:613px" %)((( 497 +non-enumerated** Representation **(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 543 543 ))) 544 -|**Value**|(% style="width:754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or 545 -| |(% style="width:754px" %)((( 546 -to a valid **value **(for non-enumerated** **Representations) 547 -))) 548 -|**Value Domain Subset / Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 549 -|**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 550 -|**Described Value Domain Subset / Described Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 551 -|**Set list**|(% style="width:754px" %)This abstraction does not exist in SDMX 499 +|(% style="width:278px" %)**Value**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or to a valid **value **(for non-enumerated** **Representations) 500 +|(% style="width:278px" %)**Value Domain Subset / Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 501 +|(% style="width:278px" %)**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 502 +|(% style="width:278px" %)**Described Value Domain Subset / Described Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 503 +|(% style="width:278px" %)**Set list**|(% style="width:613px" %)This abstraction does not exist in SDMX 552 552 553 553 The main difference between VTL and SDMX relies on the fact that the VTL artefacts for defining subsets of Value Domains do not exist in SDMX, therefore the VTL features for referring to predefined subsets are not available in SDMX. These artefacts are the Value Domain Subset (or Set), either enumerated or described, the Set List (list of values belonging to enumerated subsets) and the Data Set Component (aimed at defining the set of values that the Component of a Data Set can take, possibly a subset of the codes of Value Domain). 554 554 ... ... @@ -574,7 +574,7 @@ 574 574 575 575 The VTL data types are sub-divided in scalar types (like integers, strings, etc.), which are the types of the scalar values, and compound types (like Data Sets, Components, Rulesets, etc.), which are the types of the compound structures. See below the diagram of the VTL data types, taken from the VTL User Manual: 576 576 577 -[[image: SDMX3-0-0 SECTION6FINAL-1.0_en_e3df33ae.png||height="543" width="483"]]529 +[[image:1747388434672-948.png]] 578 578 579 579 (% class="wikigeneratedid" id="HFigure222013VTLDataTypes" %) 580 580 **Figure 22 – VTL Data Types** ... ... @@ -583,13 +583,10 @@ 583 583 584 584 The VTL basic scalar types are listed below and follow a hierarchical structure in terms of supersets/subsets (e.g. "scalar" is the superset of all the basic scalar types): 585 585 538 +[[image:1747388465321-274.png]] 586 586 587 587 **Figure 23 – VTL Basic Scalar Types** 588 588 589 -((( 590 - 591 -))) 592 - 593 593 === 12.4.2 VTL basic scalar types and SDMX data types === 594 594 595 595 The VTL assumes that a basic scalar type has a unique internal representation and can have more external representations. ... ... @@ -803,53 +803,53 @@ 803 803 804 804 The custom output formats can be specified by means of the VTL formatting mask described in the section "Type Conversion and Formatting Mask" of the VTL Reference Manual. Such a section describes the masks for the VTL basic scalar types "number", "integer", "date", "time", "time_period" and "duration" and gives examples. As for the types "string" and "boolean" the VTL conventions are extended with some other special characters as described in the following table. 805 805 806 - |(%colspan="2" %)VTL special characters for the formatting masks807 -|(% colspan="2" %) 808 -|(% colspan="2" %) Number809 -| D|one numeric digit(ifthe scientific notationisadopted, Dis only forthemantissa)810 -| E|one numeric digit (fortheexponent of thescientific notation)811 -| .(dot)|possibleseparatorbetweentheintegerandthedecimalparts.812 -| ,(comma)|possible separator between the integer and the decimal parts.813 -| | 814 -|(% colspan="2" %)Timeand duration815 -| C|century816 -| Y|year817 -| S|semester818 -| Q|quarter819 -| M|month820 -| W|week821 -| D|day822 -| h|hourdigit(bydefaulton 24hours)823 -| M|minute824 -| S|second825 -| D|decimalofsecond826 -| P|periodindicator (representationin onedigitfortheduration)827 -|P| numberofthe periodsspecifiedntheperiodindicator828 -| AM/PM|indicator ofAM/PM(e.g.am/pm for"am"or"pm")829 -|M ONTH|uppercasetextualrepresentationofthemonth(e.g.,JANUARYforJanuary)830 -| DAY|uppercase textual representation of theday(e.g.,MONDAY forMonday)831 -| Month|lowercase textual representation of themonth(e.g.,january)832 -| Day|lowercase textual representation of the month (e.g.,monday)833 -| Month|Firstcharacter uppercase,thenlowercase textual representation of the month (e.g.,January)834 -| Day|First character uppercase, then lowercase textual representation of theday using(e.g.Monday)835 -| 836 -|(% colspan="2" %)String837 -| X|anystring character838 -| Z|any string characterfrom "A" to "z"839 -| 9|any string character from "0" to "9"840 -| 841 -|(% colspan="2" %)Boolean842 -| B|Booleanusing "true"for True andfalse"for False843 -| 1|Boolean using "1" for True and "0" for False844 -| 0|Boolean using "0" for True and "1" for False845 -| 846 -|(% colspan="2" %)Otherqualifiers847 -| *|anarbitrarynumber of digits(oftheprecedingtype)848 -| +|atleastone digit (of the preceding type)849 -|( )|optional digits(specifiedwithin thebrackets)850 -| \|prefix forthespecialcharactersthat must appearin themask851 -| N|fixednumber of digitsusedinthe preceding textual representationof themonthor theday852 -| 755 +(% style="width:713.294px" %) 756 +|(% colspan="2" style="width:710px" %)VTL special characters for the formatting masks 757 +|(% colspan="2" style="width:710px" %) 758 +|(% colspan="2" style="width:710px" %)Number 759 +|D|(% style="width:486px" %)one numeric digit (if the scientific notation is adopted, D is only for the mantissa) 760 +|E|(% style="width:486px" %)one numeric digit (for the exponent of the scientific notation) 761 +|. (dot)|(% style="width:486px" %)possible separator between the integer and the decimal parts. 762 +|, (comma)|(% style="width:486px" %)possible separator between the integer and the decimal parts. 763 +| |(% style="width:486px" %) 764 +|(% colspan="2" style="width:710px" %)Time and duration 765 +|C|(% style="width:486px" %)century 766 +|Y|(% style="width:486px" %)year 767 +|S|(% style="width:486px" %)semester 768 +|Q|(% style="width:486px" %)quarter 769 +|M|(% style="width:486px" %)month 770 +|W|(% style="width:486px" %)week 771 +|D|(% style="width:486px" %)day 772 +|h|(% style="width:486px" %)hour digit (by default on 24 hours) 773 +|M|(% style="width:486px" %)minute 774 +|S|(% style="width:486px" %)second 775 +|D|(% style="width:486px" %)decimal of second 776 +|P|(% style="width:486px" %)period indicator (representation in one digit for the duration) 777 +|P|(% style="width:486px" %)number of the periods specified in the period indicator 778 +|AM/PM|(% style="width:486px" %)indicator of AM / PM (e.g. am/pm for "am" or "pm") 779 +|MONTH|(% style="width:486px" %)uppercase textual representation of the month (e.g., JANUARY for January) 780 +|DAY|(% style="width:486px" %)uppercase textual representation of the day (e.g., MONDAY for Monday) 781 +|Month|(% style="width:486px" %)lowercase textual representation of the month (e.g., january) 782 +|Day|(% style="width:486px" %)lowercase textual representation of the month (e.g., monday) 783 +|Month|(% style="width:486px" %)First character uppercase, then lowercase textual representation of the month (e.g., January) 784 +|Day|(% style="width:486px" %)First character uppercase, then lowercase textual representation of the day using (e.g. Monday) 785 +| |(% style="width:486px" %) 786 +|(% colspan="2" style="width:710px" %)String 787 +|X|(% style="width:486px" %)any string character 788 +|Z|(% style="width:486px" %)any string character from "A" to "z" 789 +|9|(% style="width:486px" %)any string character from "0" to "9" 790 +| |(% style="width:486px" %) 791 +|(% colspan="2" style="width:710px" %)Boolean 792 +|B|(% style="width:486px" %)Boolean using "true" for True and "false" for False 793 +|1|(% style="width:486px" %)Boolean using "1" for True and "0" for False 794 +|0|(% style="width:486px" %)Boolean using "0" for True and "1" for False 795 +| |(% style="width:486px" %) 796 +|(% colspan="2" style="width:710px" %)Other qualifiers 797 +|*|(% style="width:486px" %)an arbitrary number of digits (of the preceding type) 798 +|+|(% style="width:486px" %)at least one digit (of the preceding type) 799 +|( )|(% style="width:486px" %)optional digits (specified within the brackets) 800 +|\|(% style="width:486px" %)prefix for the special characters that must appear in the mask 801 +|N|(% style="width:486px" %)fixed number of digits used in the preceding textual representation of the month or the day 853 853 854 854 The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL Transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion{{footnote}}The representation given in the DSD should obviously be compatible with the VTL data type.{{/footnote}}. 855 855
- 1747388148322-387.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +23.0 KB - Content
- 1747388179021-814.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +66.2 KB - Content
- 1747388206717-256.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +10.3 KB - Content
- 1747388222879-916.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +9.9 KB - Content
- 1747388244829-693.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +7.4 KB - Content
- 1747388275998-621.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +19.2 KB - Content
- 1747388434672-948.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +43.2 KB - Content
- 1747388465321-274.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +17.9 KB - Content