Skip to Content

Changes for page 13 Structure Mapping

Last modified by Artur on 2025/09/10 11:19
From version 4.20
edited by Helena
on 2025/06/16 15:20
Change comment: There is no comment for this version
To version 4.11
edited by Helena
on 2025/06/16 14:55
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -358,7 +358,7 @@
358 358  
359 359  === 13.7.2 Numerical based datetime ===
360 360  
361 -Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]], and the **Period** which must take one of the following enumerated values:
361 +Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid SDMX Time Period, and the **Period** which must take one of the following enumerated values:
362 362  
363 363  * day
364 364  * second
... ... @@ -366,63 +366,75 @@
366 366  * microsecond
367 367  * nanosecond
368 368  
369 -(% style="width:545.294px" %)
370 -|(% style="width:268px" %)Numerical datetime systems|(% style="width:121px" %)Base|(% style="width:153px" %)Period
371 -|(% style="width:268px" %)(((
369 +|Numerical datetime systems|Base|Period
370 +|(((
372 372  Epoch Time (UNIX)
372 +
373 373  Milliseconds since 01 Jan 1970
374 -)))|(% style="width:121px" %)1970|(% style="width:153px" %)millisecond
375 -|(% style="width:268px" %)(((
374 +)))|1970|millisecond
375 +|(((
376 376  Windows System Time
377 +
377 377  Milliseconds since 01 Jan 1601
378 -)))|(% style="width:121px" %)1601|(% style="width:153px" %)millisecond
379 +)))|1601|millisecond
379 379  
380 380  The example above illustrates numerical based datetime mapping rules for two commonly used time standards.
381 381  
382 382  The model is illustrated below:
383 383  
384 -[[image:1750074994887-415.png]]
385 +[[image:1750072341491-790.jpeg]]
385 385  
386 -**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use **
387 +**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the**
387 387  
388 -=== 13.7.3 Mapping more complex time inputs ===
389 +==== TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use ====
389 389  
390 -[[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
391 +=== Mapping more complex time inputs ===
391 391  
392 -== 13.8 Using TIME_PERIOD in mapping rules ==
393 +VTL should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
393 393  
394 -The source TIME_PERIOD [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] can be used in conjunction with other input [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] to create discrete mapping rules where the output is conditional on the [[time period>>doc:sdmx:Glossary.Time period.WebHome]] value.
395 +== Using TIME_PERIOD in mapping rules ==
395 395  
396 -The main use case is setting the value of Observation [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]] in the target [[dataset>>doc:sdmx:Glossary.Data set.WebHome]].
397 +The source TIME_PERIOD Dimension can be used in conjunction with other input Dimensions to create discrete mapping rules where the output is conditional on the time period value.
397 397  
398 -(% style="width:709.294px" %)
399 -|(% style="width:103px" %)**Rule**|(% style="width:405px" %)**Source**|(% style="width:198px" %)**Target**
400 -|(% style="width:103px" %)1|(% style="width:405px" %)(((
399 +The main use case is setting the value of Observation Attributes in the target dataset.
400 +
401 +|Rule|Source|Target
402 +|1|(((
401 401  If
404 +
402 402  INDICATOR=XULADS; and TIME_PERIOD=2007.
403 -)))|(% style="width:198px" %)(((
404 -Set OBS_CONF=F
406 +)))|(((
407 +Set
408 +
409 +OBS_CONF=F
405 405  )))
406 -|(% style="width:103px" %)2|(% style="width:405px" %)(((
411 +|2|(((
407 407  If
413 +
408 408  INDICATOR=XULADS; and TIME_PERIOD=2008.
409 -)))|(% style="width:198px" %)Set OBS_CONF=F
410 -|(% style="width:103px" %)3|(% style="width:405px" %)(((
415 +)))|Set OBS_CONF=F
416 +|3|(((
411 411  If
418 +
412 412  INDICATOR=XULADS; and TIME_PERIOD=2009.
413 -)))|(% style="width:198px" %)(((
414 -Set OBS_CONF=F
420 +)))|(((
421 +Set
422 +
423 +OBS_CONF=F
415 415  )))
416 -|(% style="width:103px" %)4|(% style="width:405px" %)(((
425 +|4|(((
417 417  If
427 +
418 418  INDICATOR=XULADS; and TIME_PERIOD=2010.
419 -)))|(% style="width:198px" %)(((
420 -Set OBS_CONF=**C**
429 +)))|(((
430 +Set
431 +
432 +OBS_CONF=**C**
421 421  )))
422 422  
423 -In the example above, OBS_CONF is an Observation [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]].
435 +In the example above, OBS_CONF is an Observation Attribute.
424 424  
425 -== 13.9 Time span mapping rules using validity periods ==
437 +== 13. Time span mapping rules using validity periods ==
426 426  
427 427  Creating discrete mapping rules for each TIME_PERIOD is impractical where rules need to cover a specific span of time regardless of frequency, and for high-frequency data.
428 428  
... ... @@ -430,22 +430,25 @@
430 430  
431 431  By specifying validity periods, the example from Section 13.8 can be re-written using two rules as follows:
432 432  
433 -(% style="width:516.294px" %)
434 -|(% style="width:74px" %)Rule|(% style="width:215px" %)Source|(% style="width:223px" %)Target
435 -|(% style="width:74px" %)1|(% style="width:215px" %)(((
445 +|Rule|Source|Target
446 +|1|(((
436 436  If
448 +
437 437  INDICATOR=XULADS.
438 -Validity Period
439 -start period=2007
440 -end period=2009
441 -)))|(% style="width:223px" %)Set OBS_CONF=F
442 -|(% style="width:74px" %)2|(% style="width:215px" %)(((
450 +
451 +
452 +Validity Period start period=2007 end period=2009
453 +)))|Set OBS_CONF=F
454 +|2|(((
443 443  If
456 +
444 444  INDICATOR=XULADS.
445 -Validity Period
446 -start period=2010
447 -)))|(% style="width:223px" %)(((
458 +
459 +
460 +Validity Period start period=2010
461 +)))|(((
448 448  Set
463 +
449 449  OBS_CONF=F
450 450  )))
451 451  
... ... @@ -453,100 +453,106 @@
453 453  
454 454  In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
455 455  
456 -In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
471 +== 13. Mapping examples ==
457 457  
458 -== 13.10 Mapping examples ==
473 +=== 13. Many to one mapping (N-1) ===
459 459  
460 -=== 13.10.1 Many to one mapping (N3513 -1) ===
475 +|Source|Map To
476 +|(((
477 +**FREQ**="A"
461 461  
462 -(% style="width:542.294px" %)
463 -|(% style="width:274px" %)**Source**|(% style="width:265px" %)**Map To**
464 -|(% style="width:274px" %)(((
465 -(% style="color:#8e44ad" %)**FREQ**="A"
466 466  ADJUSTMENT="N"
480 +
467 467  **REF_AREA**="PL"
482 +
468 468  **COUNTERPART_AREA**="W0"
484 +
469 469  REF_SECTOR="S1"
486 +
470 470  COUNTERPART_SECTOR="S1"
488 +
471 471  ACCOUNTING_ENTRY="B"
490 +
472 472  STO="B5G"
473 -)))|(% style="width:265px" %)(((
474 -(% style="color:#8e44ad" %)FREQ="A"
492 +)))|(((
493 +FREQ="A"
494 +
475 475  REF_AREA="PL"
496 +
476 476  COUNTERPART_AREA="W0"
477 -INDICATOR="IND_ABC"(%%)
498 +INDICATOR="IND_ABC"
478 478  
479 479  )))
480 480  
481 -The bold [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] (% style="color:#e74c3c" %)map(%%) from source to target verbatim. The mapping simply specifies:
502 +The bold Dimensions map from source to target verbatim. The mapping simply specifies:
482 482  
483 -> FREQ => FREQ
484 -> REF_AREA=> REF_AREA
485 -> COUNTERPART_AREA=> COUNTERPART _AREA
504 +FREQ => FREQ
486 486  
487 -No [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required. The source value simply copies across unmodified.
506 +REF_AREA=> REF_AREA
488 488  
489 -The remaining [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] all (% style="color:#e74c3c" %)map(%%) to the Indicator [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. This is an example of many [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] mapping to one [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. In this case a [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required, and the mapping first describes the input 'partial key' and how this (% style="color:#e74c3c" %)maps(%%) to the target indicator:
508 +COUNTERPART_AREA=> COUNTERPART _AREA
490 490  
491 -> N:S1:S1:B:B5G => IND_ABC
510 +No Representation Mapping is required. The source value simply copies across unmodified.
492 492  
493 -Where the key sequence is based on the order specified in the (% style="color:#e74c3c" %)mapping(%%) (i.e [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]], REF_SECTOR, etc will result in the first value N being taken from [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]] as this was the first item in the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] list.
512 +The remaining Dimensions all map to the Indicator Dimension. This is an example of many Dimensions mapping to one Dimension. In this case a Representation Mapping is required, and the mapping first describes the input 'partial key' and how this maps to the target indicator:
494 494  
495 -**Note**: The key order is NOT based on the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] order of the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], as the (% style="color:#e74c3c" %)mapping(%%) needs to be resilient to the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] changing.
514 +N:S1:S1:B:B5G => IND_ABC
496 496  
497 -=== 13.10.2 Mapping other data types to Code Id ===
516 +Where the key sequence is based on the order specified in the mapping (i.e ADJUSTMENT, REF_SECTOR, etc will result in the first value N being taken from ADJUSTMENT as this was the first item in the source Dimension list.
498 498  
499 -In the case where the incoming data type is not a string and not a [[code>>doc:sdmx:Glossary.Code.WebHome]] identifier i.e. the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
518 +**Note**: The key order is NOT based on the Dimension order of the DSD, as the mapping needs to be resilient to the DSD changing.
500 500  
501 -The following [[representation>>doc:sdmx:Glossary.Representation.WebHome]] (% style="color:#e74c3c" %)mapping(%%) can be used to explicitly (% style="color:#e74c3c" %)map(%%) each [[age>>doc:sdmx:Glossary.Age.WebHome]] to an output [[code>>doc:sdmx:Glossary.Code.WebHome]].
520 +1.
521 +11.
522 +111. Mapping other data types to Code Id
502 502  
503 -(% style="width:402.294px" %)
504 -|(% style="width:197px" %)**Source Input Free Text**|(% style="width:204px" %)**Desired Output Code Id**
505 -|(% style="width:197px" %)0|(% style="width:204px" %)A
506 -|(% style="width:197px" %)1|(% style="width:204px" %)A
507 -|(% style="width:197px" %)2|(% style="width:204px" %)A
508 -|(% style="width:197px" %)3|(% style="width:204px" %)B
509 -|(% style="width:197px" %)4|(% style="width:204px" %)B
524 +In the case where the incoming data type is not a string and not a code identifier i.e. the source Dimension is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
510 510  
526 +The following representation mapping can be used to explicitly map each age to an output code.
527 +
528 +|Source Input Free Text|Desired Output Code Id
529 +|0|A
530 +|1|A
531 +|2|A
532 +|3|B
533 +|4|B
534 +
511 511  If this mapping takes advantage of regular expressions it can be expressed in two rules:
512 512  
513 -(% style="width:336.294px" %)
514 -|(% style="width:182px" %)**Regular Expression**|(% style="width:151px" %)**Desired Output**
515 -|(% style="width:182px" %)[0-2]|(% style="width:151px" %)A
516 -|(% style="width:182px" %)[3-4]|(% style="width:151px" %)B
517 517  
518 -=== 13.10.3 Observation Attributes for Time Period ===
538 +Regular Expression Desired Output
519 519  
520 -This use case is where a specific observation for a specific [[time period>>doc:sdmx:Glossary.Time period.WebHome]] has an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] value.
540 +|[0-2]|A
541 +|[3-4]|B
521 521  
522 -(% style="width:621.294px" %)
523 -|(% style="width:201px" %)Input INDICATOR|(% style="width:192px" %)Input TIME_PERIOD|(% style="width:225px" %)Output OBS_CONF
524 -|(% style="width:201px" %)XULADS|(% style="width:192px" %)2008|(% style="width:225px" %)C
525 -|(% style="width:201px" %)XULADS|(% style="width:192px" %)2009|(% style="width:225px" %)C
526 -|(% style="width:201px" %)XULADS|(% style="width:192px" %)2010|(% style="width:225px" %)C
543 +=== 13. Observation Attributes for Time Period ===
527 527  
528 -Or using a validity period on the [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping:
545 +This use case is where a specific observation for a specific time period has an attribute value.
529 529  
530 -(% style="width:629.294px" %)
531 -|(% style="width:202px" %)Input INDICATOR|(% style="width:197px" %)Valid From/ Valid To|(% style="width:227px" %) Output OBS_CONF
532 -|(% style="width:202px" %)XULADS|(% style="width:197px" %) 2008/2010|(% style="width:227px" %)С
547 +|Input INDICATOR|Input TIME_PERIOD|Output OBS_CONF
548 +|XULADS|2008|C
549 +|XULADS|2009|C
550 +|XULADS|2010|C
533 533  
534 -=== 13.10.4  Time mapping ===
552 +Or using a validity period on the Representation Mapping:
535 535  
536 -This use case is to create a [[time period>>doc:sdmx:Glossary.Time period.WebHome]] from an input that does not respect [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Time Formats.
554 +Input INDICATOR Valid From/ Valid To Output OBS_CONF
537 537  
538 -The [[Component>>doc:sdmx:Glossary.Component.WebHome]] Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details:
556 +XULADS 2008/2010 C
539 539  
540 -(% style="width:652.294px" %)
541 -|(% style="width:139px" %)Source Value|(% style="width:165px" %)Source Mapping|(% style="width:182px" %)Target Frequency|(% style="width:163px" %)Output
542 -|(% style="width:139px" %)18/07/1981|(% style="width:165px" %)dd/MM/yyyy|(% style="width:182px" %)A|(% style="width:163px" %)1981
558 +=== 13. Time mapping ===
543 543  
544 -When the target frequency is based on another target [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] value, in this example the value of the FREQ [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] in the target [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].
560 +This use case is to create a time period from an input that does not respect SDMX Time Formats.
545 545  
546 -|Source Value| Source Mapping|Target|Frequency Output
562 +The Component Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details:
547 547  
564 +|Source Value|Source Mapping|Target Frequency|Output
565 +|18/07/1981|dd/MM/yyyy|A|1981
548 548  
567 +When the target frequency is based on another target Dimension value, in this example the value of the FREQ Dimension in the target DSD.
549 549  
569 +Source Value Source Mapping Target Frequency Output
570 +
550 550  Dimension
551 551  
552 552  |18/07/1981 dd/MM/yyyy FREQ| |1981-07-18 (when FREQ=D)
1750074994887-415.png
Author
... ... @@ -1,1 +1,0 @@
1 -xwiki:XWiki.helena
Size
... ... @@ -1,1 +1,0 @@
1 -12.4 KB
Content