Skip to Content

Changes for page 13 Structure Mapping

Last modified by Artur on 2025/09/10 11:19
From version 4.12
edited by Helena
on 2025/06/16 14:56
Change comment: There is no comment for this version
To version 6.1
edited by Helena
on 2025/06/16 15:30
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -225,7 +225,7 @@
225 225  
226 226  The input 'G' matches on the last rule which is used as a catch-all or default in this example.
227 227  
228 -=== 13. Substrings ===
228 +=== 13.6.2 Substrings ===
229 229  
230 230  Substrings provide an alternative to regular expressions where the required section of an input value can be described using the number of the starting character, and the length of the substring in characters. The first character is at position 1.
231 231  
... ... @@ -381,58 +381,48 @@
381 381  
382 382  The model is illustrated below:
383 383  
384 +[[image:1750074994887-415.png]]
384 384  
385 -**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the**
386 +**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use **
386 386  
387 -==== TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use ====
388 +=== 13.7.3 Mapping more complex time inputs ===
388 388  
389 -=== Mapping more complex time inputs ===
390 +[[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
390 390  
391 -VTL should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
392 +== 13.8 Using TIME_PERIOD in mapping rules ==
392 392  
393 -== Using TIME_PERIOD in mapping rules ==
394 +The source TIME_PERIOD [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] can be used in conjunction with other input [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] to create discrete mapping rules where the output is conditional on the [[time period>>doc:sdmx:Glossary.Time period.WebHome]] value.
394 394  
395 -The source TIME_PERIOD Dimension can be used in conjunction with other input Dimensions to create discrete mapping rules where the output is conditional on the time period value.
396 +The main use case is setting the value of Observation [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]] in the target [[dataset>>doc:sdmx:Glossary.Data set.WebHome]].
396 396  
397 -The main use case is setting the value of Observation Attributes in the target dataset.
398 -
399 -|Rule|Source|Target
400 -|1|(((
398 +(% style="width:709.294px" %)
399 +|(% style="width:103px" %)**Rule**|(% style="width:405px" %)**Source**|(% style="width:198px" %)**Target**
400 +|(% style="width:103px" %)1|(% style="width:405px" %)(((
401 401  If
402 -
403 403  INDICATOR=XULADS; and TIME_PERIOD=2007.
404 -)))|(((
405 -Set
406 -
407 -OBS_CONF=F
403 +)))|(% style="width:198px" %)(((
404 +Set OBS_CONF=F
408 408  )))
409 -|2|(((
406 +|(% style="width:103px" %)2|(% style="width:405px" %)(((
410 410  If
411 -
412 412  INDICATOR=XULADS; and TIME_PERIOD=2008.
413 -)))|Set OBS_CONF=F
414 -|3|(((
409 +)))|(% style="width:198px" %)Set OBS_CONF=F
410 +|(% style="width:103px" %)3|(% style="width:405px" %)(((
415 415  If
416 -
417 417  INDICATOR=XULADS; and TIME_PERIOD=2009.
418 -)))|(((
419 -Set
420 -
421 -OBS_CONF=F
413 +)))|(% style="width:198px" %)(((
414 +Set OBS_CONF=F
422 422  )))
423 -|4|(((
416 +|(% style="width:103px" %)4|(% style="width:405px" %)(((
424 424  If
425 -
426 426  INDICATOR=XULADS; and TIME_PERIOD=2010.
427 -)))|(((
428 -Set
429 -
430 -OBS_CONF=**C**
419 +)))|(% style="width:198px" %)(((
420 +Set OBS_CONF=**C**
431 431  )))
432 432  
433 -In the example above, OBS_CONF is an Observation Attribute.
423 +In the example above, OBS_CONF is an Observation [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]].
434 434  
435 -== 13. Time span mapping rules using validity periods ==
425 +== 13.9 Time span mapping rules using validity periods ==
436 436  
437 437  Creating discrete mapping rules for each TIME_PERIOD is impractical where rules need to cover a specific span of time regardless of frequency, and for high-frequency data.
438 438  
... ... @@ -440,25 +440,22 @@
440 440  
441 441  By specifying validity periods, the example from Section 13.8 can be re-written using two rules as follows:
442 442  
443 -|Rule|Source|Target
444 -|1|(((
433 +(% style="width:516.294px" %)
434 +|(% style="width:74px" %)Rule|(% style="width:215px" %)Source|(% style="width:223px" %)Target
435 +|(% style="width:74px" %)1|(% style="width:215px" %)(((
445 445  If
446 -
447 447  INDICATOR=XULADS.
448 -
449 -
450 -Validity Period start period=2007 end period=2009
451 -)))|Set OBS_CONF=F
452 -|2|(((
438 +Validity Period
439 +start period=2007
440 +end period=2009
441 +)))|(% style="width:223px" %)Set OBS_CONF=F
442 +|(% style="width:74px" %)2|(% style="width:215px" %)(((
453 453  If
454 -
455 455  INDICATOR=XULADS.
456 -
457 -
458 -Validity Period start period=2010
459 -)))|(((
445 +Validity Period
446 +start period=2010
447 +)))|(% style="width:223px" %)(((
460 460  Set
461 -
462 462  OBS_CONF=F
463 463  )))
464 464  
... ... @@ -466,132 +466,118 @@
466 466  
467 467  In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
468 468  
469 -== 13. Mapping examples ==
456 +In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
470 470  
471 -=== 13. Many to one mapping (N-1) ===
458 +== 13.10 Mapping examples ==
472 472  
473 -|Source|Map To
474 -|(((
475 -**FREQ**="A"
460 +=== 13.10.1 Many to one mapping (N3513 -1) ===
476 476  
462 +(% style="width:542.294px" %)
463 +|(% style="width:274px" %)**Source**|(% style="width:265px" %)**Map To**
464 +|(% style="width:274px" %)(((
465 +(% style="color:#8e44ad" %)**FREQ**="A"
477 477  ADJUSTMENT="N"
478 -
479 479  **REF_AREA**="PL"
480 -
481 481  **COUNTERPART_AREA**="W0"
482 -
483 483  REF_SECTOR="S1"
484 -
485 485  COUNTERPART_SECTOR="S1"
486 -
487 487  ACCOUNTING_ENTRY="B"
488 -
489 489  STO="B5G"
490 -)))|(((
491 -FREQ="A"
492 -
473 +)))|(% style="width:265px" %)(((
474 +(% style="color:#8e44ad" %)FREQ="A"
493 493  REF_AREA="PL"
494 -
495 495  COUNTERPART_AREA="W0"
496 -INDICATOR="IND_ABC"
477 +INDICATOR="IND_ABC"(%%)
497 497  
498 498  )))
499 499  
500 -The bold Dimensions map from source to target verbatim. The mapping simply specifies:
481 +The bold [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] (% style="color:#e74c3c" %)map(%%) from source to target verbatim. The mapping simply specifies:
501 501  
502 -FREQ => FREQ
483 +> FREQ => FREQ
484 +> REF_AREA=> REF_AREA
485 +> COUNTERPART_AREA=> COUNTERPART _AREA
503 503  
504 -REF_AREA=> REF_AREA
487 +No [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required. The source value simply copies across unmodified.
505 505  
506 -COUNTERPART_AREA=> COUNTERPART _AREA
489 +The remaining [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] all (% style="color:#e74c3c" %)map(%%) to the Indicator [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. This is an example of many [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] mapping to one [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. In this case a [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required, and the mapping first describes the input 'partial key' and how this (% style="color:#e74c3c" %)maps(%%) to the target indicator:
507 507  
508 -No Representation Mapping is required. The source value simply copies across unmodified.
491 +> N:S1:S1:B:B5G => IND_ABC
509 509  
510 -The remaining Dimensions all map to the Indicator Dimension. This is an example of many Dimensions mapping to one Dimension. In this case a Representation Mapping is required, and the mapping first describes the input 'partial key' and how this maps to the target indicator:
493 +Where the key sequence is based on the order specified in the (% style="color:#e74c3c" %)mapping(%%) (i.e [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]], REF_SECTOR, etc will result in the first value N being taken from [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]] as this was the first item in the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] list.
511 511  
512 -N:S1:S1:B:B5G => IND_ABC
495 +**Note**: The key order is NOT based on the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] order of the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], as the (% style="color:#e74c3c" %)mapping(%%) needs to be resilient to the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] changing.
513 513  
514 -Where the key sequence is based on the order specified in the mapping (i.e ADJUSTMENT, REF_SECTOR, etc will result in the first value N being taken from ADJUSTMENT as this was the first item in the source Dimension list.
497 +=== 13.10.2 Mapping other data types to Code Id ===
515 515  
516 -**Note**: The key order is NOT based on the Dimension order of the DSD, as the mapping needs to be resilient to the DSD changing.
499 +In the case where the incoming data type is not a string and not a [[code>>doc:sdmx:Glossary.Code.WebHome]] identifier i.e. the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
517 517  
518 -1.
519 -11.
520 -111. Mapping other data types to Code Id
501 +The following [[representation>>doc:sdmx:Glossary.Representation.WebHome]] (% style="color:#e74c3c" %)mapping(%%) can be used to explicitly (% style="color:#e74c3c" %)map(%%) each [[age>>doc:sdmx:Glossary.Age.WebHome]] to an output [[code>>doc:sdmx:Glossary.Code.WebHome]].
521 521  
522 -In the case where the incoming data type is not a string and not a code identifier i.e. the source Dimension is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
503 +(% style="width:402.294px" %)
504 +|(% style="width:197px" %)**Source Input Free Text**|(% style="width:204px" %)**Desired Output Code Id**
505 +|(% style="width:197px" %)0|(% style="width:204px" %)A
506 +|(% style="width:197px" %)1|(% style="width:204px" %)A
507 +|(% style="width:197px" %)2|(% style="width:204px" %)A
508 +|(% style="width:197px" %)3|(% style="width:204px" %)B
509 +|(% style="width:197px" %)4|(% style="width:204px" %)B
523 523  
524 -The following representation mapping can be used to explicitly map each age to an output code.
525 -
526 -|Source Input Free Text|Desired Output Code Id
527 -|0|A
528 -|1|A
529 -|2|A
530 -|3|B
531 -|4|B
532 -
533 533  If this mapping takes advantage of regular expressions it can be expressed in two rules:
534 534  
513 +(% style="width:336.294px" %)
514 +|(% style="width:182px" %)**Regular Expression**|(% style="width:151px" %)**Desired Output**
515 +|(% style="width:182px" %)[0-2]|(% style="width:151px" %)A
516 +|(% style="width:182px" %)[3-4]|(% style="width:151px" %)B
535 535  
536 -Regular Expression Desired Output
518 +=== 13.10.3 Observation Attributes for Time Period ===
537 537  
538 -|[0-2]|A
539 -|[3-4]|B
520 +This use case is where a specific observation for a specific [[time period>>doc:sdmx:Glossary.Time period.WebHome]] has an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] value.
540 540  
541 -=== 13. Observation Attributes for Time Period ===
522 +(% style="width:621.294px" %)
523 +|(% style="width:201px" %)Input INDICATOR|(% style="width:192px" %)Input TIME_PERIOD|(% style="width:225px" %)Output OBS_CONF
524 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2008|(% style="width:225px" %)C
525 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2009|(% style="width:225px" %)C
526 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2010|(% style="width:225px" %)C
542 542  
543 -This use case is where a specific observation for a specific time period has an attribute value.
528 +Or using a validity period on the [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping:
544 544  
545 -|Input INDICATOR|Input TIME_PERIOD|Output OBS_CONF
546 -|XULADS|2008|C
547 -|XULADS|2009|C
548 -|XULADS|2010|C
530 +(% style="width:629.294px" %)
531 +|(% style="width:202px" %)Input INDICATOR|(% style="width:197px" %)Valid From/ Valid To|(% style="width:227px" %) Output OBS_CONF
532 +|(% style="width:202px" %)XULADS|(% style="width:197px" %) 2008/2010|(% style="width:227px" %)С
549 549  
550 -Or using a validity period on the Representation Mapping:
534 +=== 13.10.4 Time mapping ===
551 551  
552 -Input INDICATOR Valid From/ Valid To Output OBS_CONF
536 +This use case is to create a [[time period>>doc:sdmx:Glossary.Time period.WebHome]] from an input that does not respect [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Time Formats.
553 553  
554 -XULADS 2008/2010 C
538 +The [[Component>>doc:sdmx:Glossary.Component.WebHome]] Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details:
555 555  
556 -=== 13. Time mapping ===
540 +(% style="width:652.294px" %)
541 +|(% style="width:139px" %)Source Value|(% style="width:165px" %)Source Mapping|(% style="width:182px" %)Target Frequency|(% style="width:163px" %)Output
542 +|(% style="width:139px" %)18/07/1981|(% style="width:165px" %)dd/MM/yyyy|(% style="width:182px" %)A|(% style="width:163px" %)1981
557 557  
558 -This use case is to create a time period from an input that does not respect SDMX Time Formats.
544 +When the target frequency is based on another target [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] value, in this example the value of the FREQ [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] in the target [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]].
559 559  
560 -The Component Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details:
546 +(% style="width:658.294px" %)
547 +|(% style="width:143px" %)Source Value|(% style="width:163px" %) Source Mapping|(% style="width:176px" %)Target Dimension|(% style="width:173px" %)Frequency Output
548 +|(% style="width:143px" %)18/07/1981|(% style="width:163px" %)dd/MM/yyyy|(% style="width:176px" %)FREQ|(% style="width:173px" %)1981-07-18 (when FREQ=D)
561 561  
562 -|Source Value|Source Mapping|Target Frequency|Output
563 -|18/07/1981|dd/MM/yyyy|A|1981
550 + When the source is a numerical format.
564 564  
565 -When the target frequency is based on another target Dimension value, in this example the value of the FREQ Dimension in the target DSD.
552 +(% style="width:658.294px" %)
553 +|(% style="width:143px" %)Source Value|(% style="width:163px" %) Start Period|(% style="width:176px" %)Interval|(% style="width:176px" %)Target FREQ|(% style="width:173px" %) Output
554 +|(% style="width:143px" %)1589808220|(% style="width:163px" %)1970|(% style="width:176px" %) millisecond|(% style="width:176px" %)M|(% style="width:173px" %)2020-05
566 566  
567 -Source Value Source Mapping Target Frequency Output
568 -
569 -Dimension
570 -
571 -|18/07/1981 dd/MM/yyyy FREQ| |1981-07-18 (when FREQ=D)
572 -| When the source is a numerical format| |
573 -|Source Value Start Period Interval|(((
574 -Target
575 -
576 -FREQ
577 -)))|Output
578 -|1589808220 1970 millisecond|M|2020-05
579 -
580 580  When the source frequency is lower than the target frequency additional information 3568 can be provided for resolve to start of period, end of period, or mid period, as shown 3569 in the following example:
581 581  
582 - Source Value Source Mapping Target Frequency Output
558 +(% style="width:666.294px" %)
559 +|(% style="width:131px" %) Source Value|(% style="width:143px" %)Source Mapping|(% style="width:149px" %)Target Dimension|(% style="width:114px" %)Frequency|(% style="width:126px" %)Output
560 +|(% style="width:131px" %)1981|(% style="width:143px" %)yyyy|(% style="width:149px" %)D – End of Period|(% style="width:114px" %) |(% style="width:126px" %)1981-12-31
583 583  
584 -Dimension
562 +When the start of year is April 1^^st^^ the Structure (% style="color:#e74c3c" %)Map(%%) has YearStart=04-01:
585 585  
586 - 1981 yyyy D – End of Period 1981-12-31
564 +(% style="width:666.294px" %)
565 +|(% style="width:131px" %) Source Value|(% style="width:143px" %)Source Mapping|(% style="width:149px" %)Target Dimension|(% style="width:114px" %)Frequency|(% style="width:126px" %)Output
566 +|(% style="width:131px" %)1981|(% style="width:143px" %)yyyy|(% style="width:149px" %)D – End of Period|(% style="width:114px" %) |(% style="width:126px" %)1982-03-31
587 587  
588 -
589 -When the start of year is April 1^^st^^ the Structure Map has YearStart=04-01:
590 -
591 - Source Value Source Mapping Target Frequency Output
592 -
593 -Dimension
594 -
595 595  ----
596 596  
597 597  {{putFootnotes/}}
1750074994887-415.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +12.4 KB
Content