Skip to Content

Changes for page 13 Structure Mapping

Last modified by Artur on 2025/09/10 11:19
From version 4.11
edited by Helena
on 2025/06/16 14:55
Change comment: There is no comment for this version
To version 5.1
edited by Helena
on 2025/06/16 15:29
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -358,7 +358,7 @@
358 358  
359 359  === 13.7.2 Numerical based datetime ===
360 360  
361 -Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid SDMX Time Period, and the **Period** which must take one of the following enumerated values:
361 +Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]], and the **Period** which must take one of the following enumerated values:
362 362  
363 363  * day
364 364  * second
... ... @@ -366,75 +366,63 @@
366 366  * microsecond
367 367  * nanosecond
368 368  
369 -|Numerical datetime systems|Base|Period
370 -|(((
369 +(% style="width:545.294px" %)
370 +|(% style="width:268px" %)Numerical datetime systems|(% style="width:121px" %)Base|(% style="width:153px" %)Period
371 +|(% style="width:268px" %)(((
371 371  Epoch Time (UNIX)
372 -
373 373  Milliseconds since 01 Jan 1970
374 -)))|1970|millisecond
375 -|(((
374 +)))|(% style="width:121px" %)1970|(% style="width:153px" %)millisecond
375 +|(% style="width:268px" %)(((
376 376  Windows System Time
377 -
378 378  Milliseconds since 01 Jan 1601
379 -)))|1601|millisecond
378 +)))|(% style="width:121px" %)1601|(% style="width:153px" %)millisecond
380 380  
381 381  The example above illustrates numerical based datetime mapping rules for two commonly used time standards.
382 382  
383 383  The model is illustrated below:
384 384  
385 -[[image:1750072341491-790.jpeg]]
384 +[[image:1750074994887-415.png]]
386 386  
387 -**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the**
386 +**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use **
388 388  
389 -==== TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use ====
388 +=== 13.7.3 Mapping more complex time inputs ===
390 390  
391 -=== Mapping more complex time inputs ===
390 +[[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
392 392  
393 -VTL should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
392 +== 13.8 Using TIME_PERIOD in mapping rules ==
394 394  
395 -== Using TIME_PERIOD in mapping rules ==
394 +The source TIME_PERIOD [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] can be used in conjunction with other input [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] to create discrete mapping rules where the output is conditional on the [[time period>>doc:sdmx:Glossary.Time period.WebHome]] value.
396 396  
397 -The source TIME_PERIOD Dimension can be used in conjunction with other input Dimensions to create discrete mapping rules where the output is conditional on the time period value.
396 +The main use case is setting the value of Observation [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]] in the target [[dataset>>doc:sdmx:Glossary.Data set.WebHome]].
398 398  
399 -The main use case is setting the value of Observation Attributes in the target dataset.
400 -
401 -|Rule|Source|Target
402 -|1|(((
398 +(% style="width:709.294px" %)
399 +|(% style="width:103px" %)**Rule**|(% style="width:405px" %)**Source**|(% style="width:198px" %)**Target**
400 +|(% style="width:103px" %)1|(% style="width:405px" %)(((
403 403  If
404 -
405 405  INDICATOR=XULADS; and TIME_PERIOD=2007.
406 -)))|(((
407 -Set
408 -
409 -OBS_CONF=F
403 +)))|(% style="width:198px" %)(((
404 +Set OBS_CONF=F
410 410  )))
411 -|2|(((
406 +|(% style="width:103px" %)2|(% style="width:405px" %)(((
412 412  If
413 -
414 414  INDICATOR=XULADS; and TIME_PERIOD=2008.
415 -)))|Set OBS_CONF=F
416 -|3|(((
409 +)))|(% style="width:198px" %)Set OBS_CONF=F
410 +|(% style="width:103px" %)3|(% style="width:405px" %)(((
417 417  If
418 -
419 419  INDICATOR=XULADS; and TIME_PERIOD=2009.
420 -)))|(((
421 -Set
422 -
423 -OBS_CONF=F
413 +)))|(% style="width:198px" %)(((
414 +Set OBS_CONF=F
424 424  )))
425 -|4|(((
416 +|(% style="width:103px" %)4|(% style="width:405px" %)(((
426 426  If
427 -
428 428  INDICATOR=XULADS; and TIME_PERIOD=2010.
429 -)))|(((
430 -Set
431 -
432 -OBS_CONF=**C**
419 +)))|(% style="width:198px" %)(((
420 +Set OBS_CONF=**C**
433 433  )))
434 434  
435 -In the example above, OBS_CONF is an Observation Attribute.
423 +In the example above, OBS_CONF is an Observation [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]].
436 436  
437 -== 13. Time span mapping rules using validity periods ==
425 +== 13.9 Time span mapping rules using validity periods ==
438 438  
439 439  Creating discrete mapping rules for each TIME_PERIOD is impractical where rules need to cover a specific span of time regardless of frequency, and for high-frequency data.
440 440  
... ... @@ -442,25 +442,22 @@
442 442  
443 443  By specifying validity periods, the example from Section 13.8 can be re-written using two rules as follows:
444 444  
445 -|Rule|Source|Target
446 -|1|(((
433 +(% style="width:516.294px" %)
434 +|(% style="width:74px" %)Rule|(% style="width:215px" %)Source|(% style="width:223px" %)Target
435 +|(% style="width:74px" %)1|(% style="width:215px" %)(((
447 447  If
448 -
449 449  INDICATOR=XULADS.
450 -
451 -
452 -Validity Period start period=2007 end period=2009
453 -)))|Set OBS_CONF=F
454 -|2|(((
438 +Validity Period
439 +start period=2007
440 +end period=2009
441 +)))|(% style="width:223px" %)Set OBS_CONF=F
442 +|(% style="width:74px" %)2|(% style="width:215px" %)(((
455 455  If
456 -
457 457  INDICATOR=XULADS.
458 -
459 -
460 -Validity Period start period=2010
461 -)))|(((
445 +Validity Period
446 +start period=2010
447 +)))|(% style="width:223px" %)(((
462 462  Set
463 -
464 464  OBS_CONF=F
465 465  )))
466 466  
... ... @@ -468,132 +468,118 @@
468 468  
469 469  In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
470 470  
471 -== 13. Mapping examples ==
456 +In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
472 472  
473 -=== 13. Many to one mapping (N-1) ===
458 +== 13.10 Mapping examples ==
474 474  
475 -|Source|Map To
476 -|(((
477 -**FREQ**="A"
460 +=== 13.10.1 Many to one mapping (N3513 -1) ===
478 478  
462 +(% style="width:542.294px" %)
463 +|(% style="width:274px" %)**Source**|(% style="width:265px" %)**Map To**
464 +|(% style="width:274px" %)(((
465 +(% style="color:#8e44ad" %)**FREQ**="A"
479 479  ADJUSTMENT="N"
480 -
481 481  **REF_AREA**="PL"
482 -
483 483  **COUNTERPART_AREA**="W0"
484 -
485 485  REF_SECTOR="S1"
486 -
487 487  COUNTERPART_SECTOR="S1"
488 -
489 489  ACCOUNTING_ENTRY="B"
490 -
491 491  STO="B5G"
492 -)))|(((
493 -FREQ="A"
494 -
473 +)))|(% style="width:265px" %)(((
474 +(% style="color:#8e44ad" %)FREQ="A"
495 495  REF_AREA="PL"
496 -
497 497  COUNTERPART_AREA="W0"
498 -INDICATOR="IND_ABC"
477 +INDICATOR="IND_ABC"(%%)
499 499  
500 500  )))
501 501  
502 -The bold Dimensions map from source to target verbatim. The mapping simply specifies:
481 +The bold [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] (% style="color:#e74c3c" %)map(%%) from source to target verbatim. The mapping simply specifies:
503 503  
504 -FREQ => FREQ
483 +> FREQ => FREQ
484 +> REF_AREA=> REF_AREA
485 +> COUNTERPART_AREA=> COUNTERPART _AREA
505 505  
506 -REF_AREA=> REF_AREA
487 +No [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required. The source value simply copies across unmodified.
507 507  
508 -COUNTERPART_AREA=> COUNTERPART _AREA
489 +The remaining [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] all (% style="color:#e74c3c" %)map(%%) to the Indicator [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. This is an example of many [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] mapping to one [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. In this case a [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required, and the mapping first describes the input 'partial key' and how this (% style="color:#e74c3c" %)maps(%%) to the target indicator:
509 509  
510 -No Representation Mapping is required. The source value simply copies across unmodified.
491 +> N:S1:S1:B:B5G => IND_ABC
511 511  
512 -The remaining Dimensions all map to the Indicator Dimension. This is an example of many Dimensions mapping to one Dimension. In this case a Representation Mapping is required, and the mapping first describes the input 'partial key' and how this maps to the target indicator:
493 +Where the key sequence is based on the order specified in the (% style="color:#e74c3c" %)mapping(%%) (i.e [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]], REF_SECTOR, etc will result in the first value N being taken from [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]] as this was the first item in the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] list.
513 513  
514 -N:S1:S1:B:B5G => IND_ABC
495 +**Note**: The key order is NOT based on the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] order of the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], as the (% style="color:#e74c3c" %)mapping(%%) needs to be resilient to the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] changing.
515 515  
516 -Where the key sequence is based on the order specified in the mapping (i.e ADJUSTMENT, REF_SECTOR, etc will result in the first value N being taken from ADJUSTMENT as this was the first item in the source Dimension list.
497 +=== 13.10.2 Mapping other data types to Code Id ===
517 517  
518 -**Note**: The key order is NOT based on the Dimension order of the DSD, as the mapping needs to be resilient to the DSD changing.
499 +In the case where the incoming data type is not a string and not a [[code>>doc:sdmx:Glossary.Code.WebHome]] identifier i.e. the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
519 519  
520 -1.
521 -11.
522 -111. Mapping other data types to Code Id
501 +The following [[representation>>doc:sdmx:Glossary.Representation.WebHome]] (% style="color:#e74c3c" %)mapping(%%) can be used to explicitly (% style="color:#e74c3c" %)map(%%) each [[age>>doc:sdmx:Glossary.Age.WebHome]] to an output [[code>>doc:sdmx:Glossary.Code.WebHome]].
523 523  
524 -In the case where the incoming data type is not a string and not a code identifier i.e. the source Dimension is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
503 +(% style="width:402.294px" %)
504 +|(% style="width:197px" %)**Source Input Free Text**|(% style="width:204px" %)**Desired Output Code Id**
505 +|(% style="width:197px" %)0|(% style="width:204px" %)A
506 +|(% style="width:197px" %)1|(% style="width:204px" %)A
507 +|(% style="width:197px" %)2|(% style="width:204px" %)A
508 +|(% style="width:197px" %)3|(% style="width:204px" %)B
509 +|(% style="width:197px" %)4|(% style="width:204px" %)B
525 525  
526 -The following representation mapping can be used to explicitly map each age to an output code.
527 -
528 -|Source Input Free Text|Desired Output Code Id
529 -|0|A
530 -|1|A
531 -|2|A
532 -|3|B
533 -|4|B
534 -
535 535  If this mapping takes advantage of regular expressions it can be expressed in two rules:
536 536  
513 +(% style="width:336.294px" %)
514 +|(% style="width:182px" %)**Regular Expression**|(% style="width:151px" %)**Desired Output**
515 +|(% style="width:182px" %)[0-2]|(% style="width:151px" %)A
516 +|(% style="width:182px" %)[3-4]|(% style="width:151px" %)B
537 537  
538 -Regular Expression Desired Output
518 +=== 13.10.3 Observation Attributes for Time Period ===
539 539  
540 -|[0-2]|A
541 -|[3-4]|B
520 +This use case is where a specific observation for a specific [[time period>>doc:sdmx:Glossary.Time period.WebHome]] has an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] value.
542 542  
543 -=== 13. Observation Attributes for Time Period ===
522 +(% style="width:621.294px" %)
523 +|(% style="width:201px" %)Input INDICATOR|(% style="width:192px" %)Input TIME_PERIOD|(% style="width:225px" %)Output OBS_CONF
524 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2008|(% style="width:225px" %)C
525 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2009|(% style="width:225px" %)C
526 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2010|(% style="width:225px" %)C
544 544  
545 -This use case is where a specific observation for a specific time period has an attribute value.
528 +Or using a validity period on the [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping:
546 546  
547 -|Input INDICATOR|Input TIME_PERIOD|Output OBS_CONF
548 -|XULADS|2008|C
549 -|XULADS|2009|C
550 -|XULADS|2010|C
530 +(% style="width:629.294px" %)
531 +|(% style="width:202px" %)Input INDICATOR|(% style="width:197px" %)Valid From/ Valid To|(% style="width:227px" %) Output OBS_CONF
532 +|(% style="width:202px" %)XULADS|(% style="width:197px" %) 2008/2010|(% style="width:227px" %)С
551 551  
552 -Or using a validity period on the Representation Mapping:
534 +=== 13.10.4  Time mapping ===
553 553  
554 -Input INDICATOR Valid From/ Valid To Output OBS_CONF
536 +This use case is to create a [[time period>>doc:sdmx:Glossary.Time period.WebHome]] from an input that does not respect [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Time Formats.
555 555  
556 -XULADS 2008/2010 C
538 +The [[Component>>doc:sdmx:Glossary.Component.WebHome]] Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details:
557 557  
558 -=== 13. Time mapping ===
540 +(% style="width:652.294px" %)
541 +|(% style="width:139px" %)Source Value|(% style="width:165px" %)Source Mapping|(% style="width:182px" %)Target Frequency|(% style="width:163px" %)Output
542 +|(% style="width:139px" %)18/07/1981|(% style="width:165px" %)dd/MM/yyyy|(% style="width:182px" %)A|(% style="width:163px" %)1981
559 559  
560 -This use case is to create a time period from an input that does not respect SDMX Time Formats.
544 +When the target frequency is based on another target [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] value, in this example the value of the FREQ [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] in the target [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].
561 561  
562 -The Component Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details:
546 +(% style="width:658.294px" %)
547 +|(% style="width:143px" %)Source Value|(% style="width:163px" %) Source Mapping|(% style="width:176px" %)Target Dimension|(% style="width:173px" %)Frequency Output
548 +|(% style="width:143px" %)18/07/1981|(% style="width:163px" %)dd/MM/yyyy|(% style="width:176px" %)FREQ|(% style="width:173px" %)1981-07-18 (when FREQ=D)
563 563  
564 -|Source Value|Source Mapping|Target Frequency|Output
565 -|18/07/1981|dd/MM/yyyy|A|1981
550 + When the source is a numerical format.
566 566  
567 -When the target frequency is based on another target Dimension value, in this example the value of the FREQ Dimension in the target DSD.
552 +(% style="width:658.294px" %)
553 +|(% style="width:143px" %)Source Value|(% style="width:163px" %) Start Period|(% style="width:176px" %)Interval|(% style="width:176px" %)Target FREQ|(% style="width:173px" %) Output
554 +|(% style="width:143px" %)1589808220|(% style="width:163px" %)1970|(% style="width:176px" %) millisecond|(% style="width:176px" %)M|(% style="width:173px" %)2020-05
568 568  
569 -Source Value Source Mapping Target Frequency Output
570 -
571 -Dimension
572 -
573 -|18/07/1981 dd/MM/yyyy FREQ| |1981-07-18 (when FREQ=D)
574 -| When the source is a numerical format| |
575 -|Source Value Start Period Interval|(((
576 -Target
577 -
578 -FREQ
579 -)))|Output
580 -|1589808220 1970 millisecond|M|2020-05
581 -
582 582  When the source frequency is lower than the target frequency additional information 3568 can be provided for resolve to start of period, end of period, or mid period, as shown 3569 in the following example:
583 583  
584 - Source Value Source Mapping Target Frequency Output
558 +(% style="width:666.294px" %)
559 +|(% style="width:131px" %) Source Value|(% style="width:143px" %)Source Mapping|(% style="width:149px" %)Target Dimension|(% style="width:114px" %)Frequency|(% style="width:126px" %)Output
560 +|(% style="width:131px" %)1981|(% style="width:143px" %)yyyy|(% style="width:149px" %)D – End of Period|(% style="width:114px" %) |(% style="width:126px" %)1981-12-31
585 585  
586 -Dimension
562 +When the start of year is April 1^^st^^ the Structure (% style="color:#e74c3c" %)Map(%%) has YearStart=04-01:
587 587  
588 - 1981 yyyy D – End of Period 1981-12-31
564 +(% style="width:666.294px" %)
565 +|(% style="width:131px" %) Source Value|(% style="width:143px" %)Source Mapping|(% style="width:149px" %)Target Dimension|(% style="width:114px" %)Frequency|(% style="width:126px" %)Output
566 +|(% style="width:131px" %)1981|(% style="width:143px" %)yyyy|(% style="width:149px" %)D – End of Period|(% style="width:114px" %) |(% style="width:126px" %)1982-03-31
589 589  
590 -
591 -When the start of year is April 1^^st^^ the Structure Map has YearStart=04-01:
592 -
593 - Source Value Source Mapping Target Frequency Output
594 -
595 -Dimension
596 -
597 597  ----
598 598  
599 599  {{putFootnotes/}}
1750074994887-415.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +12.4 KB
Content