Changes for page 10 Validation and Transformation Language (VTL)
Last modified by Helena on 2025/09/10 11:19
Summary
-
Page properties (2 modified, 0 added, 0 removed)
-
Attachments (0 modified, 3 added, 0 removed)
Details
- Page properties
-
- Author
-
... ... @@ -1,1 +1,1 @@ 1 -xwiki:XWiki. arturkryazhev1 +xwiki:XWiki.helena - Content
-
... ... @@ -53,7 +53,7 @@ 53 53 54 54 The **[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] prefix** is “urn:sdmx:org”, always the same for all [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]. 55 55 56 -The ** [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]-package-name **is the concatenation of the string** **“[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].infomodel.” with the package-name which the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] belongs to. For example, for referencing a [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] the SDMX-IM-package-name is “sdmx.infomodel.datastructure”, because the class [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] belongs to the package “datastructure”.56 +The **SDMX-IM-package-name **is the concatenation of the string** **“[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].infomodel.” with the package-name which the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] belongs to. For example, for referencing a [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] the SDMX-IM-package-name is “sdmx.infomodel.datastructure”, because the class [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] belongs to the package “datastructure”. 57 57 58 58 The **class-name** is the name of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] object class which the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] object belongs to (e.g., for referencing a [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] the class-name is “[[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]”). The [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] can reference [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] that belong to the classes [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, (% style="color:#e74c3c" %)[[Concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), ConceptScheme, Codelist. 59 59 ... ... @@ -89,7 +89,7 @@ 89 89 The URN can be abbreviated by omitting the parts that are not essential for the identification of the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] or that can be deduced from other available information, including the context in which the invocation is made. The possible abbreviations are described below. 90 90 91 91 * The **SDMXPrefix** can be omitted for all the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] objects, because it is a prefixed string (urn:[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]:org), always the same for [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] objects. 92 -* The ** [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]-package-name **can be omitted as well because it can be deduced from the class-name that follows it (the table of the [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]] packages and classes that allows this deduction is in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] 2.1 Standards - Section 5 - Registry Specifications, paragraph 6.2.3). In particular, considering the object classes of the [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] that [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] can reference, the package is:92 +* The **SDMX-IM-package-name **can be omitted as well because it can be deduced from the class-name that follows it (the table of the [[SDMX-IM>>doc:sdmx:Glossary.SDMX Information Model.WebHome]] packages and classes that allows this deduction is in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] 2.1 Standards - Section 5 - Registry Specifications, paragraph 6.2.3). In particular, considering the object classes of the [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] that [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] can reference, the package is: 93 93 ** “datastructure” for the classes [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, 94 94 ** “conceptscheme” for the classes (% style="color:#e74c3c" %)[[Concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) and ConceptScheme 95 95 ** “codelist” for the class Codelist. ... ... @@ -98,12 +98,12 @@ 98 98 * As for the **maintainedobject-id**, this is essential in some cases while in other cases it can be omitted: o if the referenced [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] is a [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], which is a maintainable class, the maintainedobject-id is the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]-id and obviously cannot be omitted; 99 99 ** if the referenced [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] is a [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute, which are not maintainable and belong to the DataStructure maintainable class, the maintainedobject-id is the dataStructure-id and can be omitted, given that these [[components>>doc:sdmx:Glossary.Component.WebHome]] are always invoked within the invocation of a [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], whose dataStructure-id can be deduced from the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] structural definitions; 100 100 ** if the referenced [[artefa>>doc:sdmx:Glossary.Artefact.WebHome]] 101 -** 101 +** 102 102 *** ct is a (% style="color:#e74c3c" %)[[Concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), which is not maintainable and belong to the ConceptScheme maintainable class,,, ,,the maintained object is the conceptScheme-id and cannot be omitted; 103 103 *** if the referenced [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] is a ConceptScheme, which is a,, ,,maintainable class,,, ,,the maintained object is the conceptScheme-id and obviously cannot be omitted; 104 104 *** if the referenced [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] is a [[Codelist>>doc:sdmx:Glossary.Code list.WebHome]], which is a maintainable class, the maintainedobject-id is the [[codelist>>doc:sdmx:Glossary.Code list.WebHome]]-id and obviously cannot be omitted. 105 -** 106 -*** When the maintainedobject-id is omitted, the **maintainedobject- (% style="color:#e74c3c" %)version(%%)** is omitted too. When the maintainedobject-id is not omitted and the maintainedobject-(% style="color:#e74c3c" %)version(%%)is omitted, the(% style="color:#e74c3c" %)version(%%)1.0 is assumed by default.105 +** 106 +*** When the maintainedobject-id is omitted, the **maintainedobject-version** is omitted too. When the maintainedobject-id is not omitted and the maintainedobject-version is omitted, the version 1.0 is assumed by default. 107 107 *** As said, the **container-object-id** does not apply to the classes that can be referenced in [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] transformations, therefore is not present in their URN 108 108 *** The **object-id** does not exist for the [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] belonging to the [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], ConceptScheme and [[Codelist>>doc:sdmx:Glossary.Code list.WebHome]] classes, while it exists and cannot be omitted for the [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] belonging to the classes [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], MeasureDimension, TimeDimension, PrimaryMeasure, DataAttribute and [[Concept>>doc:sdmx:Glossary.Concept.WebHome]], as for them the object-id is the main identifier of the [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] 109 109 ... ... @@ -133,10 +133,8 @@ 133 133 134 134 The corresponding fully abbreviated reference, if made from a transformation scheme belonging to AG, would become simply: 135 135 136 -S 136 +SECTOR 137 137 138 -ECTOR 139 - 140 140 For example, the transformation for renaming the [[component>>doc:sdmx:Glossary.Component.WebHome]] SECTOR of the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] DF1 into SEC can be written as{{footnote}}The result DFR(1.0) is be equal to DF1(1.0) save that the component SECTOR is called SEC{{/footnote}}: 141 141 142 142 ‘DFR(1.0)’ := ‘DF1(1.0)’ [rename SECTOR to SEC] ... ... @@ -270,10 +270,8 @@ 270 270 * The value of the PrimaryMeasure of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] observation belonging to the set above and having MeasureDimension=Cj becomes the value of the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[Measure>>doc:sdmx:Glossary.Measure.WebHome]] Cj 271 271 * For the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] DataAttributes depending on the MeasureDimension, the value of the DataAttribute DA of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] observation belonging to the set above and having MeasureDimension=Cj becomes the value of the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]] DA_Cj 272 272 273 -==== 10.3.3.3 Fr ==== 271 +==== 10.3.3.3 From SDMX DataAttributes to VTL Measures ==== 274 274 275 -==== om SDMX DataAttributes to VTL Measures ==== 276 - 277 277 * In some cases it may happen that the DataAttributes of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] DataStructure need to be managed as [[Measures>>doc:sdmx:Glossary.Measure.WebHome]] in [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]]. Therefore, a variant of both the methods above consists in transforming all the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] DataAttributes in [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[Measures>>doc:sdmx:Glossary.Measure.WebHome]]. When DataAttributes are converted to [[Measures>>doc:sdmx:Glossary.Measure.WebHome]], the two methods above are called Basic_A2M and Pivot_A2M (the suffix “A2M” stands for [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]] to [[Measures>>doc:sdmx:Glossary.Measure.WebHome]]). Obviously, the resulting [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] data structure is, in general, multi-[[measure>>doc:sdmx:Glossary.Measure.WebHome]] and does not contain [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]]. 278 278 279 279 The Basic_A2M and Pivot_A2M behaves respectively like the Basic and Pivot methods, except that the final [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[components>>doc:sdmx:Glossary.Component.WebHome]], which according to the Basic and Pivot methods would have had the role of [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]], assume instead the role of [[Measure>>doc:sdmx:Glossary.Measure.WebHome]]. ... ... @@ -353,7 +353,7 @@ 353 353 354 354 Like in the general case, a MeasureDimension [[component>>doc:sdmx:Glossary.Component.WebHome]] called “measure_name” would be added to the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] DataStructure and would have just one possible [[measure>>doc:sdmx:Glossary.Measure.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), corresponding to the unique [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[measure>>doc:sdmx:Glossary.Measure.WebHome]]. The original [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[component>>doc:sdmx:Glossary.Component.WebHome]] would not become a [[Component>>doc:sdmx:Glossary.Component.WebHome]] in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] data structure. The value of the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[measure>>doc:sdmx:Glossary.Measure.WebHome]] would be assigned to the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] PrimaryMeasure called “obs_value”. 355 355 356 -In any case, the resulting [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] definitions must be compliant with the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] consistency rules. For example, the possible (% style="color:#e74c3c" %)[[Concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] MeasureDimension need to be listed in a [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] ConceptScheme, with proper id, agency and [[(% style="color: rgb(231,76, 60);color: rgb(231, 76, 60)" %)version>>doc:Glossary.Version.WebHome]](%%); moreover, the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] must have the assignmentStatus, which does not exist in [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]], the attributeRelationship for the DataAttributes and so on.352 +In any case, the resulting [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] definitions must be compliant with the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] consistency rules. For example, the possible (% style="color:#e74c3c" %)[[Concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] MeasureDimension need to be listed in a [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] ConceptScheme, with proper id, agency and [[(% style="color:#e74c3c" %)version>>doc:Glossary.Version.WebHome]](%%); moreover, the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] must have the assignmentStatus, which does not exist in [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]], the attributeRelationship for the DataAttributes and so on. 357 357 358 358 ==== 10.3.4.3 From VTL Measures to SDMX Data Attributes** ** ==== 359 359 ... ... @@ -488,12 +488,15 @@ 488 488 489 489 As said, it is assumed that these [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] derived [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] have the TIME_PERIOD as the only identifier. In the mapping from [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] to SMDX, the [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] INDICATOR and COUNTRY are added to the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] data structure on order to obtain the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] one, with the following values respectively: 490 490 491 -[[image:1747859458410-183.png||height="170" width="663"]] 492 492 488 +[[image:1752871956541-370.png]] 489 + 493 493 It should be noted that the application of this many-to-one mapping from [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] to [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is equivalent to an appropriate sequence of [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] Transformations. These use the [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] operator “calc” to add the proper [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] identifiers (in the example, INDICATOR and COUNTRY) and to assign to them the proper values and the operator “union” in order to obtain the final [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[dataset>>doc:sdmx:Glossary.Data set.WebHome]] (in the example DF2(1.0)), that can be mapped one-to-one to the homonymous [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]]. Following the same example, these [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] transformations would be: 494 494 495 -[[image:17 47859612718-454.png||height="451" width="602"]]492 +[[image:1752871982030-664.png]] 496 496 494 +[[image:1752872007793-429.png]] 495 + 497 497 In other words, starting from the [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] explicitly calculated through [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] (in the example ‘DF2(1.0)/GDPPERCAPITA.USA’ and so on), the first step consists in calculating other (non-persistent) [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] (in the example DF2bis_GDPPERCAPITA_USA and so on) by adding the identifiers INDICATOR and COUNTRY with the desired values (//INDICATORvalue// and //COUNTRYvalue)//. Finally, all these non-persistent [[data sets>>doc:sdmx:Glossary.Data set.WebHome]] are united and give the final result DF2(1.0){{footnote}}The result is persistent in this example but it can be also non persistent if needed.{{/footnote}}, which can be mapped one-to-one to the homonymous [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] having the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] [[components>>doc:sdmx:Glossary.Component.WebHome]] TIME_PERIOD, INDICATOR and COUNTRY. 498 498 499 499 Therefore, mapping different [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] having the same data structure to different parts of a [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], i.e. in the direction from [[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] to [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], through the ordered concatenation notation is equivalent to a proper use of the operators “calc” and “union” on such [[datasets>>doc:sdmx:Glossary.Data set.WebHome]].{{footnote}}In case the ordered concatenation notation from VTL to SDMX is used, the set of transformations described above is implicitly performed; therefore, in order to test the overall compliance of the VTL program to the VTL consistency rules, these implicit transformations have to be considered as part of the VTL program even if they are not explicitly coded.{{/footnote}}
- 1752871956541-370.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +42.5 KB - Content
- 1752871982030-664.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +97.8 KB - Content
- 1752872007793-429.png
-
- Author
-
... ... @@ -1,0 +1,1 @@ 1 +xwiki:XWiki.helena - Size
-
... ... @@ -1,0 +1,1 @@ 1 +11.3 KB - Content