Changes for page 12 Validation and Transformation Language (VTL)
Last modified by Helena on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -14,8 +14,10 @@ 14 14 15 15 The VTL language can be applied to SDMX artefacts by mapping the SDMX IM model artefacts to the model artefacts that VTL can manipulate{{footnote}}In this chapter, in order to distinguish VTL and SDMX model artefacts, the VTL ones are written in the Arial font while the SDMX ones in Courier New{{/footnote}}. Thus, the SDMX artefacts can be used in VTL as inputs and/or outputs of Transformations. It is important to be aware that the artefacts do not always have the same names in the SDMX and VTL IMs, nor do they always have the same meaning. The more evident example is given by the SDMX Dataset and the VTL "Data Set", which do not correspond one another: as a matter of fact, the VTL "Data Set" maps to the SDMX "Dataflow", while the SDMX "Dataset" has no explicit mapping to VTL (such an abstraction is not needed in the definition of VTL Transformations). A SDMX "Dataset", however, is an instance of a SDMX "Dataflow" and can be the artefact on which the VTL transformations are executed (i.e., the Transformations are defined on Dataflows and are applied to Dataflow instances that can be Datasets). 16 16 17 -The VTL programs (Transformation Schemes) are represented in SDMX through the TransformationScheme maintainable class which is composed of Transformation (nameable artefact). Each Transformation assigns the outcome of the evaluation of a VTL expression to a result.17 +The VTL programs (Transformation Schemes) are represented in SDMX through the TransformationScheme maintainable class which is composed of 18 18 19 +Transformation (nameable artefact). Each Transformation assigns the outcome of the evaluation of a VTL expression to a result. 20 + 19 19 This section does not explain the VTL language or any of the content published in the VTL guides. Rather, this is a description of how the VTL can be used in the SDMX context and applied to SDMX artefacts. 20 20 21 21 == 12.2 References to SDMX artefacts from VTL statements == ... ... @@ -241,7 +241,7 @@ 241 241 242 242 AttributeRelationship is not specified (i.e. the DataAttribute does not depend on any DimensionComponent and therefore is at data set level), or if it refers to a set (or a group) of dimensions which does not include the MeasureDimension; 243 243 244 -* 246 +* 245 245 ** Otherwise, if, according to the SDMX AttributeRelationship, the values of the DataAttribute depend on the MeasureDimension, the SDMX DataAttribute is mapped to one VTL Attribute for each possible Code of the SDMX MeasureDimension. By default, the names of the VTL Attributes are obtained by concatenating the name of the SDMX DataAttribute and the names of the correspondent Code of the MeasureDimension separated by underscore. For example, if the SDMX DataAttribute is named DA and the possible Codes of the SDMX MeasureDimension are named C1, C2, …, Cn, then the corresponding VTL Attributes will be named DA_C1, DA_C2, …, DA_Cn (if different names are desired, they can be achieved afterwards by renaming the Attributes through VTL operators). 246 246 ** Like in the Basic mapping, the resulting VTL Attributes are considered as dependent on all the VTL identifiers (i.e. "at data point / observation level"), because VTL does not have the SDMX notion of Attribute Relationship. 247 247 ... ... @@ -599,7 +599,7 @@ 599 599 [[image:SDMX 3-0-0 SECTION 6 FINAL-1.0_en_e3df33ae.png||height="543" width="483"]] 600 600 601 601 (% class="wikigeneratedid" id="HFigure222013VTLDataTypes" %) 602 - **Figure 22 – VTL Data Types**604 +Figure 22 – VTL Data Types 603 603 604 604 The VTL scalar types are in turn subdivided in basic scalar types, which are elementary (not defined in term of other data types) and Value Domain and Set scalar types, which are defined in terms of the basic scalar types. 605 605 ... ... @@ -606,7 +606,7 @@ 606 606 The VTL basic scalar types are listed below and follow a hierarchical structure in terms of supersets/subsets (e.g. "scalar" is the superset of all the basic scalar types): 607 607 608 608 609 - **Figure 23 – VTL Basic Scalar Types**611 +Figure 23 – VTL Basic Scalar Types 610 610 611 611 ((( 612 612 ... ... @@ -634,159 +634,204 @@ 634 634 635 635 The following table describes the default mapping for converting from the SDMX data types to the VTL basic scalar types. 636 636 637 -(% style="width:823.294px" %) 638 -|(% style="width:509px" %)**SDMX data type (BasicComponentDataType)**|(% style="width:312px" %)**Default VTL basic scalar type** 639 -|(% style="width:509px" %)((( 639 +|SDMX data type (BasicComponentDataType)|Default VTL basic scalar type 640 +|((( 640 640 String 642 + 641 641 (string allowing any character) 642 -)))| (%style="width:312px" %)string643 -|( % style="width:509px" %)(((644 +)))|string 645 +|((( 644 644 Alpha 647 + 645 645 (string which only allows A-z) 646 -)))| (%style="width:312px" %)string647 -|( % style="width:509px" %)(((649 +)))|string 650 +|((( 648 648 AlphaNumeric 652 + 649 649 (string which only allows A-z and 0-9) 650 -)))| (%style="width:312px" %)string651 -|( % style="width:509px" %)(((654 +)))|string 655 +|((( 652 652 Numeric 657 + 653 653 (string which only allows 0-9, but is not numeric so that is can having leading zeros) 654 -)))| (%style="width:312px" %)string655 -|( % style="width:509px" %)(((659 +)))|string 660 +|((( 656 656 BigInteger 662 + 657 657 (corresponds to XML Schema xs:integer datatype; infinite set of integer values) 658 -)))| (% style="width:312px" %)integer659 -|( % style="width:509px" %)(((664 +)))|integer 665 +|((( 660 660 Integer 661 -(corresponds to XML Schema xs:int datatype; between -2147483648 and +2147483647 (inclusive)) 662 -)))|(% style="width:312px" %)integer 663 -|(% style="width:509px" %)((( 667 + 668 +(corresponds to XML Schema xs:int datatype; between -2147483648 and +2147483647 669 + 670 +(inclusive)) 671 +)))|integer 672 +|((( 664 664 Long 665 -(corresponds to XML Schema xs:long datatype; between -9223372036854775808 and +9223372036854775807 (inclusive)) 666 -)))|(% style="width:312px" %)integer 667 -|(% style="width:509px" %)((( 674 + 675 +(corresponds to XML Schema xs:long datatype; between -9223372036854775808 and 676 + 677 ++9223372036854775807 (inclusive)) 678 +)))|integer 679 +|((( 668 668 Short 681 + 669 669 (corresponds to XML Schema xs:short datatype; between -32768 and -32767 (inclusive)) 670 -)))| (% style="width:312px" %)integer671 -| (% style="width:509px" %)Decimal (corresponds to XML Schema xs:decimal datatype; subset of real numbers that can be represented as decimals)|(% style="width:312px" %)number672 -|( % style="width:509px" %)(((683 +)))|integer 684 +|Decimal (corresponds to XML Schema xs:decimal datatype; subset of real numbers that can be represented as decimals)|number 685 +|((( 673 673 Float 687 + 674 674 (corresponds to XML Schema xs:float datatype; patterned after the IEEE single-precision 32-bit floating point type) 675 -)))| (% style="width:312px" %)number676 -|( % style="width:509px" %)(((689 +)))|number 690 +|((( 677 677 Double 692 + 678 678 (corresponds to XML Schema xs:double datatype; patterned after the IEEE double-precision 64-bit floating point type) 679 -)))| (% style="width:312px" %)number680 -|( % style="width:509px" %)(((694 +)))|number 695 +|((( 681 681 Boolean 682 -(corresponds to the XML Schema xs:boolean datatype; support the mathematical concept of binary-valued logic: {true, false}) 683 -)))|(% style="width:312px" %)boolean 684 684 685 -(% style="width:822.294px" %) 686 -|(% colspan="2" style="width:507px" %)((( 698 +(corresponds to the XML Schema xs:boolean datatype; support the mathematical concept of 699 + 700 +binary-valued logic: {true, false}) 701 +)))|boolean 702 + 703 +| |(% colspan="2" %)((( 687 687 URI 705 + 688 688 (corresponds to the XML Schema xs:anyURI; absolute or relative Uniform Resource Identifier Reference) 689 -)))|(% colspan=" 1"style="width:311px"%)string690 -|(% colspan="2" style="width:507px"%)(((707 +)))|(% colspan="2" %)string 708 +| |(% colspan="2" %)((( 691 691 Count 710 + 692 692 (an integer following a sequential pattern, increasing by 1 for each occurrence) 693 -)))|(% colspan=" 1"style="width:311px"%)integer694 -|(% colspan="2" style="width:507px"%)(((712 +)))|(% colspan="2" %)integer 713 +| |(% colspan="2" %)((( 695 695 InclusiveValueRange 715 + 696 696 (decimal number within a closed interval, whose bounds are specified in the SDMX representation by the facets minValue and maxValue) 697 -)))|(% colspan=" 1"style="width:311px"%)number698 -|(% colspan="2" style="width:507px"%)(((717 +)))|(% colspan="2" %)number 718 +| |(% colspan="2" %)((( 699 699 ExclusiveValueRange 720 + 700 700 (decimal number within an open interval, whose bounds are specified in the SDMX representation by the facets minValue and maxValue) 701 -)))|(% colspan=" 1"style="width:311px"%)number702 -|(% colspan="2" style="width:507px"%)(((722 +)))|(% colspan="2" %)number 723 +| |(% colspan="2" %)((( 703 703 Incremental 725 + 704 704 (decimal number the increased by a specific interval (defined by the interval facet), which is typically enforced outside of the XML validation) 705 -)))|(% colspan=" 1"style="width:311px"%)number706 -|(% colspan="2" style="width:507px"%)(((727 +)))|(% colspan="2" %)number 728 +| |(% colspan="2" %)((( 707 707 ObservationalTimePeriod 730 + 708 708 (superset of StandardTimePeriod and TimeRange) 709 -)))|(% colspan=" 1"style="width:311px"%)time710 -|(% colspan="2" style="width:507px"%)(((732 +)))|(% colspan="2" %)time 733 +| |(% colspan="2" %)((( 711 711 StandardTimePeriod 712 -(superset of BasicTimePeriod and ReportingTimePeriod) 713 -)))|(% colspan="1" style="width:311px" %)time 714 -|(% colspan="2" style="width:507px" %)((( 735 + 736 +(superset of BasicTimePeriod and 737 + 738 +ReportingTimePeriod) 739 +)))|(% colspan="2" %)time 740 +| |(% colspan="2" %)((( 715 715 BasicTimePeriod 742 + 716 716 (superset of GregorianTimePeriod and DateTime) 717 -)))|(% colspan=" 1"style="width:311px"%)date718 -|(% colspan="2" style="width:507px"%)(((744 +)))|(% colspan="2" %)date 745 +| |(% colspan="2" %)((( 719 719 GregorianTimePeriod 747 + 720 720 (superset of GregorianYear, GregorianYearMonth, and GregorianDay) 721 -)))|(% colspan=" 1"style="width:311px"%)date722 -|(% colspan="2" style="width:507px"%)GregorianYear (YYYY)|(% colspan="1"style="width:311px"%)date723 -|(% colspan="2" style="width:507px"%)GregorianYearMonth / GregorianMonth (YYYY-MM)|(% colspan="1"style="width:311px"%)date724 -|(% colspan="2" style="width:507px"%)GregorianDay (YYYY-MM-DD)|(% colspan="1"style="width:311px"%)date725 -|(% colspan="2" style="width:507px"%)(((749 +)))|(% colspan="2" %)date 750 +| |(% colspan="2" %)GregorianYear (YYYY)|(% colspan="2" %)date 751 +| |(% colspan="2" %)GregorianYearMonth / GregorianMonth (YYYY-MM)|(% colspan="2" %)date 752 +| |(% colspan="2" %)GregorianDay (YYYY-MM-DD)|(% colspan="2" %)date 753 +| |(% colspan="2" %)((( 726 726 ReportingTimePeriod 727 -(superset of RepostingYear, ReportingSemester, ReportingTrimester, ReportingQuarter, ReportingMonth, ReportingWeek, ReportingDay) 728 -)))|(% colspan="1" style="width:311px" %)time_period 729 -|(% colspan="2" style="width:507px" %)((( 755 + 756 +(superset of RepostingYear, ReportingSemester, 757 + 758 +ReportingTrimester, ReportingQuarter, 759 + 760 +ReportingMonth, ReportingWeek, ReportingDay) 761 +)))|(% colspan="2" %)time_period 762 +| |(% colspan="2" %)((( 730 730 ReportingYear 764 + 731 731 (YYYY-A1 – 1 year period) 732 -)))|(% colspan=" 1"style="width:311px"%)time_period733 -|(% colspan="2" style="width:507px"%)(((766 +)))|(% colspan="2" %)time_period 767 +| |(% colspan="2" %)((( 734 734 ReportingSemester 769 + 735 735 (YYYY-Ss – 6 month period) 736 -)))|(% colspan=" 1"style="width:311px"%)time_period737 -|(% colspan="2" style="width:507px"%)(((771 +)))|(% colspan="2" %)time_period 772 +| |(% colspan="2" %)((( 738 738 ReportingTrimester 774 + 739 739 (YYYY-Tt – 4 month period) 740 -)))|(% colspan=" 1"style="width:311px"%)time_period741 -|(% colspan="2" style="width:507px"%)(((776 +)))|(% colspan="2" %)time_period 777 +| |(% colspan="2" %)((( 742 742 ReportingQuarter 779 + 743 743 (YYYY-Qq – 3 month period) 744 -)))|(% colspan=" 1"style="width:311px"%)time_period745 -|(% colspan="2" style="width:507px"%)(((781 +)))|(% colspan="2" %)time_period 782 +| |(% colspan="2" %)((( 746 746 ReportingMonth 784 + 747 747 (YYYY-Mmm – 1 month period) 748 -)))|(% colspan="1" style="width:311px" %)time_period 749 -|(% colspan="2" style="width:507px" %)ReportingWeek|(% colspan="1" style="width:311px" %)time_period 750 -|(% colspan="1" style="width:507px" %)(YYYY-Www – 7 day period; following ISO 8601 definition of a week in a year)|(% colspan="2" style="width:312px" %) 751 -|(% colspan="1" style="width:507px" %)((( 786 +)))|(% colspan="2" %)time_period 787 +| |(% colspan="2" %)ReportingWeek|(% colspan="2" %)time_period 788 +| |(% colspan="2" %) |(% colspan="2" %) 789 +| |(% colspan="2" %) |(% colspan="2" %) 790 +|(% colspan="2" %)(YYYY-Www – 7 day period; following ISO 8601 definition of a week in a year)|(% colspan="2" %) | 791 +|(% colspan="2" %)((( 752 752 ReportingDay 793 + 753 753 (YYYY-Dddd – 1 day period) 754 -)))|(% colspan="2" style="width:312px"%)time_period755 -|(% colspan=" 1"style="width:507px"%)(((795 +)))|(% colspan="2" %)time_period| 796 +|(% colspan="2" %)((( 756 756 DateTime 798 + 757 757 (YYYY-MM-DDThh:mm:ss) 758 -)))|(% colspan="2" style="width:312px"%)date759 -|(% colspan=" 1"style="width:507px"%)(((800 +)))|(% colspan="2" %)date| 801 +|(% colspan="2" %)((( 760 760 TimeRange 803 + 761 761 (YYYY-MM-DD(Thh:mm:ss)?/<duration>) 762 -)))|(% colspan="2" style="width:312px"%)time763 -|(% colspan=" 1"style="width:507px"%)(((805 +)))|(% colspan="2" %)time| 806 +|(% colspan="2" %)((( 764 764 Month 808 + 765 765 (~-~-MM; speicifies a month independent of a year; e.g. February is black history month in the United States) 766 -)))|(% colspan="2" style="width:312px"%)string767 -|(% colspan=" 1"style="width:507px"%)(((810 +)))|(% colspan="2" %)string| 811 +|(% colspan="2" %)((( 768 768 MonthDay 813 + 769 769 (~-~-MM-DD; specifies a day within a month independent of a year; e.g. Christmas is December 25^^th^^; used to specify reporting year start day) 770 -)))|(% colspan="2" style="width:312px"%)string771 -|(% colspan=" 1"style="width:507px"%)(((815 +)))|(% colspan="2" %)string| 816 +|(% colspan="2" %)((( 772 772 Day 818 + 773 773 (~-~--DD; specifies a day independent of a month or year; e.g. the 15^^th^^ is payday) 774 -)))|(% colspan="2" style="width:312px"%)string775 -|(% colspan=" 1"style="width:507px"%)(((820 +)))|(% colspan="2" %)string| 821 +|(% colspan="2" %)((( 776 776 Time 823 + 777 777 (hh:mm:ss; time independent of a date; e.g. coffee break is at 10:00 AM) 778 -)))|(% colspan="2" style="width:312px"%)string779 -|(% colspan=" 1"style="width:507px"%)(((825 +)))|(% colspan="2" %)string| 826 +|(% colspan="2" %)((( 780 780 Duration 828 + 781 781 (corresponds to XML Schema xs:duration datatype) 782 -)))|(% colspan="2" style="width:312px"%)duration783 -|(% colspan=" 1"style="width:507px"%)XHTML|(% colspan="2"style="width:312px"%)Metadata type – not applicable784 -|(% colspan=" 1"style="width:507px"%)KeyValues|(% colspan="2"style="width:312px"%)Metadata type – not applicable785 -|(% colspan=" 1"style="width:507px"%)IdentifiableReference|(% colspan="2"style="width:312px"%)Metadata type – not applicable786 -|(% colspan=" 1"style="width:507px"%)DataSetReference|(% colspan="2"style="width:312px"%)Metadata type – not applicable830 +)))|(% colspan="2" %)duration| 831 +|(% colspan="2" %)XHTML|(% colspan="2" %)Metadata type – not applicable| 832 +|(% colspan="2" %)KeyValues|(% colspan="2" %)Metadata type – not applicable| 833 +|(% colspan="2" %)IdentifiableReference|(% colspan="2" %)Metadata type – not applicable| 834 +|(% colspan="2" %)DataSetReference|(% colspan="2" %)Metadata type – not applicable| 787 787 788 -(% class="wikigeneratedid" id="HFigure142013MappingsfromSDMXdatatypestoVTLBasicScalarTypes" %) 789 -**Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types** 836 +==== Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types ==== 790 790 791 791 When VTL takes in input SDMX artefacts, it is assumed that a type conversion according to the table above always happens. In case a different VTL basic scalar type is desired, it can be achieved in the VTL program taking in input the default VTL basic scalar type above and applying to it the VTL type conversion features (see the implicit and explicit type conversion and the "cast" operator in the VTL Reference Manual). 792 792 ... ... @@ -794,32 +794,39 @@ 794 794 795 795 The following table describes the default conversion from the VTL basic scalar types to the SDMX data types . 796 796 797 -(% style="width:1073.29px" %) 798 -|(% style="width:207px" %)((( 799 -**VTL basic scalar type** 800 -)))|(% style="width:462px" %)((( 801 -**Default SDMX data type (BasicComponentDataType)** 802 -)))|(% style="width:402px" %)**Default output format** 803 -|(% style="width:207px" %)String|(% style="width:462px" %)String|(% style="width:402px" %)Like XML (xs:string) 804 -|(% style="width:207px" %)Number|(% style="width:462px" %)Float|(% style="width:402px" %)Like XML (xs:float) 805 -|(% style="width:207px" %)Integer|(% style="width:462px" %)Integer|(% style="width:402px" %)Like XML (xs:int) 806 -|(% style="width:207px" %)Date|(% style="width:462px" %)DateTime|(% style="width:402px" %)YYYY-MM-DDT00:00:00Z 807 -|(% style="width:207px" %)Time|(% style="width:462px" %)StandardTimePeriod|(% style="width:402px" %)<date>/<date> (as defined above) 808 -|(% style="width:207px" %)time_period|(% style="width:462px" %)((( 844 +|((( 845 +VTL basic 846 + 847 +scalar type 848 +)))|((( 849 +Default SDMX data type 850 + 851 +(BasicComponentDataType 852 + 853 +) 854 +)))|Default output format 855 +|String|String|Like XML (xs:string) 856 +|Number|Float|Like XML (xs:float) 857 +|Integer|Integer|Like XML (xs:int) 858 +|Date|DateTime|YYYY-MM-DDT00:00:00Z 859 +|Time|StandardTimePeriod|<date>/<date> (as defined above) 860 +|time_period|((( 809 809 ReportingTimePeriod 862 + 810 810 (StandardReportingPeriod) 811 -)))|( % style="width:402px" %)(((864 +)))|((( 812 812 YYYY-Pppp 866 + 813 813 (according to SDMX ) 814 814 ))) 815 -| (% style="width:207px" %)Duration|(% style="width:462px" %)Duration|(% style="width:402px" %)(((869 +|Duration|Duration|((( 816 816 Like XML (xs:duration) 871 + 817 817 PnYnMnDTnHnMnS 818 818 ))) 819 -| (% style="width:207px" %)Boolean|(% style="width:462px" %)Boolean|(% style="width:402px" %)Like XML (xs:boolean) with the values "true" or "false"874 +|Boolean|Boolean|Like XML (xs:boolean) with the values "true" or "false" 820 820 821 -(% class="wikigeneratedid" id="HFigure142013MappingsfromSDMXdatatypestoVTLBasicScalarTypes-1" %) 822 -**Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types** 876 +==== Figure 14 – Mappings from SDMX data types to VTL Basic Scalar Types ==== 823 823 824 824 In case a different default conversion is desired, it can be achieved through the CustomTypeScheme and CustomType artefacts (see also the section Transformations and Expressions of the SDMX information model). 825 825 ... ... @@ -873,7 +873,7 @@ 873 873 |N|fixed number of digits used in the preceding textual representation of the month or the day 874 874 | | 875 875 876 -The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL Transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion {{footnote}}The representationgiven in theDSDshouldobviouslybecompatible withtheVTLdata type.{{/footnote}}.930 +The default conversion, either standard or customized, can be used to deduce automatically the representation of the components of the result of a VTL Transformation. In alternative, the representation of the resulting SDMX Dataflow can be given explicitly by providing its DataStructureDefinition. In other words, the representation specified in the DSD, if available, overrides any default conversion^^[[(% class="wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink wikiinternallink" %)^^42^^>>path:#sdfootnote42sym||name="sdfootnote42anc"]](%%)^^. 877 877 878 878 === 12.4.5 Null Values === 879 879 ... ... @@ -891,8 +891,10 @@ 891 891 892 892 A different format can be specified in the attribute "vtlLiteralFormat" of the CustomType artefact (see also the section Transformations and Expressions of the SDMX information model). 893 893 894 -Like in the case of the conversion of NULLs described in the previous paragraph, the overriding assumption is applied, for a certain VTL basic scalar type, if a value is found for the vtlLiteralFormat attribute of the CustomType of such VTL basic scalar type. The overriding assumption is applied for all the literals of a related VTL TransformationScheme.948 +Like in the case of the conversion of NULLs described in the previous paragraph, the overriding assumption is applied, for a certain VTL basic scalar type, if a value is found for the vtlLiteralFormat attribute of the CustomType of such VTL basic scalar type. The overriding assumption is applied for all the literals of a related VTL 895 895 950 +TransformationScheme. 951 + 896 896 In case a literal is operand of a VTL Cast operation, the format specified in the Cast overrides all the possible otherwise specified formats. 897 897 898 898 {{putFootnotes/}}