Changes for page 12 Validation and Transformation Language (VTL)
Last modified by Helena on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 6 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -348,7 +348,7 @@ 348 348 The mapping table is the following: 349 349 350 350 (% style="width:689.294px" %) 351 -|(% style="width:344px" %)VTL|(% style="width:341px" %)SDMX 351 +|(% style="width:344px" %)**VTL**|(% style="width:341px" %)**SDMX** 352 352 |(% style="width:344px" %)(Simple) Identifier|(% style="width:341px" %)Dimension 353 353 |(% style="width:344px" %)(Time) Identifier|(% style="width:341px" %)TimeDimension 354 354 |(% style="width:344px" %)Some Measures|(% style="width:341px" %)Measure ... ... @@ -408,26 +408,14 @@ 408 408 409 409 SDMX Dataflow having INDICATOR=//INDICATORvalue //and COUNTRY=// COUNTRYvalue//. For example, the VTL dataset ‘DF1(1.0.0)/POPULATION.USA’ would contain all the observations of DF1(1.0.0) having INDICATOR = POPULATION and COUNTRY = USA. 410 410 411 -In order to obtain the data structure of these VTL Data Sets from the SDMX one, it is assumed that the SDMX DimensionComponents on which the mapping is based are dropped, i.e. not maintained in the VTL data structure; this is possible because their values are fixed for each one of the invoked VTL Data Sets{{footnote}}If these DimensionComponents would not be dropped, the various VTL Data Sets resulting from this kind of mapping would have non-matching values for the Identifiers corresponding to the mapping Dimensions (e.g. POPULATION and COUNTRY). As a consequence, taking into account that the typical binary VTL operations at dataset level (+, -, *, / and so on) are executed on the observations having matching values for the identifiers, it would not be possible to compose the resulting VTL datasets one another (e.g. it would not be possible to calculate the population ratio between USA and CANADA).{{/footnote}}. After that, the mapping method from SDMX to VTL specified for the Dataflow DF1(1.0.0) is applied (i.e. 411 +In order to obtain the data structure of these VTL Data Sets from the SDMX one, it is assumed that the SDMX DimensionComponents on which the mapping is based are dropped, i.e. not maintained in the VTL data structure; this is possible because their values are fixed for each one of the invoked VTL Data Sets{{footnote}}If these DimensionComponents would not be dropped, the various VTL Data Sets resulting from this kind of mapping would have non-matching values for the Identifiers corresponding to the mapping Dimensions (e.g. POPULATION and COUNTRY). As a consequence, taking into account that the typical binary VTL operations at dataset level (+, -, *, / and so on) are executed on the observations having matching values for the identifiers, it would not be possible to compose the resulting VTL datasets one another (e.g. it would not be possible to calculate the population ratio between USA and CANADA).{{/footnote}}. After that, the mapping method from SDMX to VTL specified for the Dataflow DF1(1.0.0) is applied (i.e. basic, pivot …). 412 412 413 -basi c, pivot…).413 +In the example above, for all the datasets of the kind ‘DF1(1.0.0)///INDICATORvalue//.//COUNTRYvalue//’, the dimensions INDICATOR and COUNTRY would be dropped so that the data structure of all the resulting VTL Data Sets would have the identifier TIME_PERIOD only. 414 414 415 -In the example above, for all the datasets of the kind 416 - 417 -‘DF1(1.0.0)///INDICATORvalue//.//COUNTRYvalue//’, the dimensions INDICATOR and COUNTRY would be dropped so that the data structure of all the resulting VTL Data Sets would have the identifier TIME_PERIOD only. 418 - 419 419 It should be noted that the desired VTL Data Sets (i.e. of the kind ‘DF1(1.0.0)/// INDICATORvalue//.//COUNTRYvalue//’) can be obtained also by applying the VTL operator “**sub**” (subspace) to the Dataflow DF1(1.0.0), like in the following VTL expression: 420 420 421 - ‘DF1(1.0.0)/POPULATION.USA’ :=417 +[[image:1747388275998-621.png]] 422 422 423 -DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA” ]; 424 - 425 -‘DF1(1.0.0)/POPULATION.CANADA’ := 426 - 427 -DF1(1.0.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“CANADA” ]; 428 - 429 -… … … 430 - 431 431 In fact the VTL operator “sub” has exactly the same behaviour. Therefore, mapping different parts of a SDMX Dataflow to different VTL Data Sets in the direction from SDMX to VTL through the ordered concatenation notation is equivalent to a proper use of the operator “**sub**” on such a Dataflow.{{footnote}}In case the ordered concatenation notation is used, the VTL Transformation described above, e.g. ‘DF1(1.0)/POPULATION.USA’ := DF1(1.0) [ sub INDICATOR=“POPULATION”, COUNTRY=“USA”], is implicitly executed. In order to test the overall compliance of the VTL program to the VTL consistency rules, it has to be considered as part of the VTL program even if it is not explicitly coded.{{/footnote}} 432 432 433 433 In the direction from SDMX to VTL it is allowed to omit the value of one or more ... ... @@ -438,10 +438,8 @@ 438 438 439 439 This is equivalent to the application of the VTL “sub” operator only to the identifier //INDICATOR//: 440 440 441 - ‘DF1(1.0.0)/POPULATION.’ :=429 +[[image:1747388244829-693.png]] 442 442 443 -DF1(1.0.0) [ sub INDICATOR=“POPULATION” ]; 444 - 445 445 Therefore the VTL Data Set ‘DF1(1.0.0)/POPULATION.’ would have the identifiers COUNTRY and TIME_PERIOD. 446 446 447 447 Heterogeneous invocations of the same Dataflow are allowed, i.e. omitting different ... ... @@ -467,54 +467,18 @@ 467 467 468 468 Some examples follow, for some specific values of INDICATOR and COUNTRY: 469 469 470 - ‘DF2(1.0.0)/GDPPERCAPITA.USA’ <- expression11; ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ <- expression12;456 +[[image:1747388222879-916.png]] 471 471 472 - … … …458 +[[image:1747388206717-256.png]] 473 473 474 -‘DF2(1.0.0)/POPGROWTH.USA’ <- expression21; 475 - 476 -‘DF2(1.0.0)/POPGROWTH.CANADA’ <- expression22; 477 - 478 -… … … 479 - 480 480 As said, it is assumed that these VTL derived Data Sets have the TIME_PERIOD as the only identifier. In the mapping from VTL to SMDX, the Dimensions INDICATOR and COUNTRY are added to the VTL data structure on order to obtain the SDMX one, with the following values respectively: 481 481 482 - VTL dataset INDICATOR value COUNTRY value462 +[[image:1747388148322-387.png]] 483 483 484 -‘DF2(1.0.0)/GDPPERCAPITA.USA’ GDPPERCAPITA USA 485 - 486 -‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ GDPPERCAPITA CANADA … … … 487 - 488 -‘DF2(1.0.0)/POPGROWTH.USA’ POPGROWTH USA 489 - 490 -‘DF2(1.0.0)/POPGROWTH.CANADA’ POPGROWTH CANADA 491 - 492 -… … … 493 - 494 494 It should be noted that the application of this many-to-one mapping from VTL to SDMX is equivalent to an appropriate sequence of VTL Transformations. These use the VTL operator “calc” to add the proper VTL identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final VTL dataset (in the example DF2(1.0.0)), that can be mapped oneto-one to the homonymous SDMX Dataflow. Following the same example, these VTL Transformations would be: 495 495 496 - DF2bis_GDPPERCAPITA_USA:= ‘DF2(1.0.0)/GDPPERCAPITA.USA’ [calc identifier INDICATOR := ”GDPPERCAPITA”, identifier COUNTRY := ”USA”];466 +[[image:1747388179021-814.png]] 497 497 498 -DF2bis_GDPPERCAPITA_CANADA := ‘DF2(1.0.0)/GDPPERCAPITA.CANADA’ [calc identifier INDICATOR:=”GDPPERCAPITA”, identifier COUNTRY:=”CANADA”]; … … … 499 - 500 -DF2bis_POPGROWTH_USA := ‘DF2(1.0.0)/POPGROWTH.USA’ 501 - 502 -[calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”USA”]; 503 - 504 -DF2bis_POPGROWTH_CANADA’ := ‘DF2(1.0.0)/POPGROWTH.CANADA’ [calc identifier INDICATOR := ”POPGROWTH”, identifier COUNTRY := ”CANADA”]; … … … 505 - 506 -DF2(1.0) <- UNION (DF2bis_GDPPERCAPITA_USA’, 507 - 508 -DF2bis_GDPPERCAPITA_CANADA’, 509 - 510 -… , 511 - 512 -DF2bis_POPGROWTH_USA’, 513 - 514 -DF2bis_POPGROWTH_CANADA’ 515 - 516 -…); 517 - 518 518 In other words, starting from the datasets explicitly calculated through VTL (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) VTL datasets (in the example 519 519 520 520 DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent Data Sets are united and give the final result DF2(1.0){{footnote}}The result is persistent in this example but it can be also non persistent if needed.{{/footnote}}, which can be mapped one-to-one to the homonymous SDMX Dataflow having the dimension components TIME_PERIOD, INDICATOR and COUNTRY. ... ... @@ -527,38 +527,30 @@ 527 527 528 528 With reference to the VTL “model for Variables and Value domains”, the following additional mappings have to be considered: 529 529 530 -(% style="width: 1170.29px" %)531 -|**VTL**|(% style="width: 754px" %)**SDMX**532 -|**Data Set Component**|(% style="width: 754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}}533 -|**Represented Variable**|(% style="width: 754px" %)(((480 +(% style="width:895.294px" %) 481 +|(% style="width:278px" %)**VTL**|(% style="width:613px" %)**SDMX** 482 +|(% style="width:278px" %)**Data Set Component**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a Component (either a DimensionComponent or a Measure or a DataAttribute) belonging to one specific Dataflow{{footnote}}Through SDMX Constraints, it is possible to specify the values that a Component of a Dataflow can assume.{{/footnote}} 483 +|(% style="width:278px" %)**Represented Variable**|(% style="width:613px" %)((( 534 534 **Concept** with a definite 535 535 536 536 Representation 537 537 ))) 538 -|**Value Domain**|(% style="width:754px" %)((( 539 -**Representation** (see the Structure 540 - 541 -Pattern in the Base Package) 488 +|(% style="width:278px" %)**Value Domain**|(% style="width:613px" %)((( 489 +**Representation** (see the Structure Pattern in the Base Package) 542 542 ))) 543 -|**Enumerated Value Domain / Code List**|(% style="width:754px" %)**Codelist** 544 -|**Code**|(% style="width:754px" %)((( 545 -**Code** (for enumerated 546 - 547 -DimensionComponent, Measure, DataAttribute) 491 +|(% style="width:278px" %)**Enumerated Value Domain / 492 +Code List**|(% style="width:613px" %)**Codelist** 493 +|(% style="width:278px" %)**Code**|(% style="width:613px" %)((( 494 +**Code** (for enumerated DimensionComponent, Measure, DataAttribute) 548 548 ))) 549 -|**Described Value Domain**|(% style="width:754px" %)((( 550 -non-enumerated** Representation** 551 - 552 -(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 496 +|(% style="width:278px" %)**Described Value Domain**|(% style="width:613px" %)((( 497 +non-enumerated** Representation **(having Facets / ExtendedFacets, see the Structure Pattern in the Base Package) 553 553 ))) 554 -|**Value**|(% style="width:754px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or 555 -| |(% style="width:754px" %)((( 556 -to a valid **value **(for non-enumerated** **Representations) 557 -))) 558 -|**Value Domain Subset / Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 559 -|**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 560 -|**Described Value Domain Subset / Described Set**|(% style="width:754px" %)This abstraction does not exist in SDMX 561 -|**Set list**|(% style="width:754px" %)This abstraction does not exist in SDMX 499 +|(% style="width:278px" %)**Value**|(% style="width:613px" %)Although this abstraction exists in SDMX, it does not have an explicit definition and correspond to a **Code** of a Codelist (for enumerated Representations) or to a valid **value **(for non-enumerated** **Representations) 500 +|(% style="width:278px" %)**Value Domain Subset / Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 501 +|(% style="width:278px" %)**Enumerated Value Domain Subset / Enumerated Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 502 +|(% style="width:278px" %)**Described Value Domain Subset / Described Set**|(% style="width:613px" %)This abstraction does not exist in SDMX 503 +|(% style="width:278px" %)**Set list**|(% style="width:613px" %)This abstraction does not exist in SDMX 562 562 563 563 The main difference between VTL and SDMX relies on the fact that the VTL artefacts for defining subsets of Value Domains do not exist in SDMX, therefore the VTL features for referring to predefined subsets are not available in SDMX. These artefacts are the Value Domain Subset (or Set), either enumerated or described, the Set List (list of values belonging to enumerated subsets) and the Data Set Component (aimed at defining the set of values that the Component of a Data Set can take, possibly a subset of the codes of Value Domain). 564 564
- 1747388148322-387.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +23.0 KB - Content
- 1747388179021-814.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +66.2 KB - Content
- 1747388206717-256.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +10.3 KB - Content
- 1747388222879-916.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +9.9 KB - Content
- 1747388244829-693.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +7.4 KB - Content
- 1747388275998-621.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +19.2 KB - Content