Skip to Content

Changes for page 13 Structure Mapping

Last modified by Artur on 2025/09/10 11:19
From version 4.10
edited by Helena
on 2025/06/16 14:54
Change comment: There is no comment for this version
To version 4.17
edited by Helena
on 2025/06/16 15:14
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -319,30 +319,29 @@
319 319  The following pattern letters are defined (all other characters from 'A' to 'Z' and from 'a' to 'z' are reserved):
320 320  
321 321  (% style="width:850.294px" %)
322 -|(% style="width:182px" %)Letter|(% style="width:327px" %)Date or Time Component|(% style="width:180px" %)Presentation|(% style="width:157px" %)Examples
323 -|(% style="width:182px" %)G|(% style="width:327px" %)Era designator|(% style="width:180px" %)Text|(% style="width:157px" %)AD
324 -|(% style="width:182px" %)yy|(% style="width:327px" %)Year short (upper case is Year of Week{{footnote}}yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years{{/footnote}})|(% style="width:180px" %)Year|(% style="width:157px" %)96
325 -|(% style="width:182px" %)yyyy|(% style="width:327px" %)Year Full (upper case is Year of Week)|(% style="width:180px" %)Year|(% style="width:157px" %)1996
326 -|(% style="width:182px" %)MM|(% style="width:327px" %)Month number in year starting with 1|(% style="width:180px" %)Month|(% style="width:157px" %)07
327 -|(% style="width:182px" %)MMM|(% style="width:327px" %)Month name short|(% style="width:180px" %)Month|(% style="width:157px" %)Jul
328 -|(% style="width:182px" %)MMMM|(% style="width:327px" %)Month name full|(% style="width:180px" %)Month|(% style="width:157px" %)July
329 -|(% style="width:182px" %)ww|(% style="width:327px" %)Week in year|(% style="width:180px" %)Number|(% style="width:157px" %)27
330 -|(% style="width:182px" %)W|(% style="width:327px" %)Week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2
331 -|(% style="width:182px" %)DD|(% style="width:327px" %)Day in year|(% style="width:180px" %)Number|(% style="width:157px" %)189
332 -|(% style="width:182px" %)dd|(% style="width:327px" %)Day in month|(% style="width:180px" %)Number|(% style="width:157px" %)10
333 -|(% style="width:182px" %)F|(% style="width:327px" %)Day of week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2
334 -|(% style="width:182px" %)E|(% style="width:327px" %)Day name in week|(% style="width:180px" %)Text|(% style="width:157px" %)Tuesday; Tue
335 -|(% style="width:182px" %)U|(% style="width:327px" %)Day number of week (1 = Monday, ..., 7 = Sunday)|(% style="width:180px" %)Number|(% style="width:157px" %)1
336 -|(% style="width:182px" %)HH|(% style="width:327px" %)Hour in day (0-23)|(% style="width:180px" %)Number|(% style="width:157px" %)0
337 -|(% style="width:182px" %)kk|(% style="width:327px" %)Hour in day (1-24)|(% style="width:180px" %)Number|(% style="width:157px" %)24
338 -|(% style="width:182px" %)KK|(% style="width:327px" %)Hour in am/pm (0-11)|(% style="width:180px" %)Number|(% style="width:157px" %)0
339 -|(% style="width:182px" %)hh|(% style="width:327px" %)Hour in am/pm (1-12)|(% style="width:180px" %)Number|(% style="width:157px" %)12
340 -|(% style="width:182px" %)mm|(% style="width:327px" %)Minute in hour|(% style="width:180px" %)Number|(% style="width:157px" %)30
341 -|(% style="width:182px" %)ss|(% style="width:327px" %)Second in minute|(% style="width:180px" %)Number|(% style="width:157px" %)55
342 -|(% style="width:182px" %)S|(% style="width:327px" %)Millisecond|(% style="width:180px" %)Number|(% style="width:157px" %)978
343 -|(% style="width:182px" %)n|(% style="width:327px" %)(((
322 +|(% style="width:125px" %)**Letter**|(% style="width:385px" %)**Date or Time Component**|(% style="width:180px" %)**Presentation**|(% style="width:157px" %)**Examples**
323 +|(% style="width:125px" %)G|(% style="width:385px" %)Era designator|(% style="width:180px" %)Text|(% style="width:157px" %)AD
324 +|(% style="width:125px" %)yy|(% style="width:385px" %)Year short (upper case is Year of Week{{footnote}}yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years{{/footnote}})|(% style="width:180px" %)Year|(% style="width:157px" %)96
325 +|(% style="width:125px" %)yyyy|(% style="width:385px" %)Year Full (upper case is Year of Week)|(% style="width:180px" %)Year|(% style="width:157px" %)1996
326 +|(% style="width:125px" %)MM|(% style="width:385px" %)Month number in year starting with 1|(% style="width:180px" %)Month|(% style="width:157px" %)07
327 +|(% style="width:125px" %)MMM|(% style="width:385px" %)Month name short|(% style="width:180px" %)Month|(% style="width:157px" %)Jul
328 +|(% style="width:125px" %)MMMM|(% style="width:385px" %)Month name full|(% style="width:180px" %)Month|(% style="width:157px" %)July
329 +|(% style="width:125px" %)ww|(% style="width:385px" %)Week in year|(% style="width:180px" %)Number|(% style="width:157px" %)27
330 +|(% style="width:125px" %)W|(% style="width:385px" %)Week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2
331 +|(% style="width:125px" %)DD|(% style="width:385px" %)Day in year|(% style="width:180px" %)Number|(% style="width:157px" %)189
332 +|(% style="width:125px" %)dd|(% style="width:385px" %)Day in month|(% style="width:180px" %)Number|(% style="width:157px" %)10
333 +|(% style="width:125px" %)F|(% style="width:385px" %)Day of week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2
334 +|(% style="width:125px" %)E|(% style="width:385px" %)Day name in week|(% style="width:180px" %)Text|(% style="width:157px" %)Tuesday; Tue
335 +|(% style="width:125px" %)U|(% style="width:385px" %)Day number of week (1 = Monday, ..., 7 = Sunday)|(% style="width:180px" %)Number|(% style="width:157px" %)1
336 +|(% style="width:125px" %)HH|(% style="width:385px" %)Hour in day (0-23)|(% style="width:180px" %)Number|(% style="width:157px" %)0
337 +|(% style="width:125px" %)kk|(% style="width:385px" %)Hour in day (1-24)|(% style="width:180px" %)Number|(% style="width:157px" %)24
338 +|(% style="width:125px" %)KK|(% style="width:385px" %)Hour in am/pm (0-11)|(% style="width:180px" %)Number|(% style="width:157px" %)0
339 +|(% style="width:125px" %)hh|(% style="width:385px" %)Hour in am/pm (1-12)|(% style="width:180px" %)Number|(% style="width:157px" %)12
340 +|(% style="width:125px" %)mm|(% style="width:385px" %)Minute in hour|(% style="width:180px" %)Number|(% style="width:157px" %)30
341 +|(% style="width:125px" %)ss|(% style="width:385px" %)Second in minute|(% style="width:180px" %)Number|(% style="width:157px" %)55
342 +|(% style="width:125px" %)S|(% style="width:385px" %)Millisecond|(% style="width:180px" %)Number|(% style="width:157px" %)978
343 +|(% style="width:125px" %)n|(% style="width:385px" %)(((
344 344  Number of periods, used after a SDMX
345 -
346 346  Frequency Identifier such as M, Q, D (month, quarter, day)
347 347  )))|(% style="width:180px" %)Number|(% style="width:157px" %)12
348 348  
... ... @@ -359,7 +359,7 @@
359 359  
360 360  === 13.7.2 Numerical based datetime ===
361 361  
362 -Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid SDMX Time Period, and the **Period** which must take one of the following enumerated values:
361 +Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[Time Period>>doc:sdmx:Glossary.Time period.WebHome]], and the **Period** which must take one of the following enumerated values:
363 363  
364 364  * day
365 365  * second
... ... @@ -367,75 +367,63 @@
367 367  * microsecond
368 368  * nanosecond
369 369  
370 -|Numerical datetime systems|Base|Period
371 -|(((
369 +(% style="width:545.294px" %)
370 +|(% style="width:268px" %)Numerical datetime systems|(% style="width:121px" %)Base|(% style="width:153px" %)Period
371 +|(% style="width:268px" %)(((
372 372  Epoch Time (UNIX)
373 -
374 374  Milliseconds since 01 Jan 1970
375 -)))|1970|millisecond
376 -|(((
374 +)))|(% style="width:121px" %)1970|(% style="width:153px" %)millisecond
375 +|(% style="width:268px" %)(((
377 377  Windows System Time
378 -
379 379  Milliseconds since 01 Jan 1601
380 -)))|1601|millisecond
378 +)))|(% style="width:121px" %)1601|(% style="width:153px" %)millisecond
381 381  
382 382  The example above illustrates numerical based datetime mapping rules for two commonly used time standards.
383 383  
384 384  The model is illustrated below:
385 385  
386 -[[image:1750072341491-790.jpeg]]
384 +[[image:1750074994887-415.png]]
387 387  
388 -**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the**
386 +**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use **
389 389  
390 -==== TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use ====
388 +=== 13.7.3 Mapping more complex time inputs ===
391 391  
392 -=== Mapping more complex time inputs ===
390 +[[VTL>>doc:sdmx:Glossary.Validation and transformation language.WebHome]] should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
393 393  
394 -VTL should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods.
392 +== 13.8 Using TIME_PERIOD in mapping rules ==
395 395  
396 -== Using TIME_PERIOD in mapping rules ==
394 +The source TIME_PERIOD [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] can be used in conjunction with other input [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] to create discrete mapping rules where the output is conditional on the [[time period>>doc:sdmx:Glossary.Time period.WebHome]] value.
397 397  
398 -The source TIME_PERIOD Dimension can be used in conjunction with other input Dimensions to create discrete mapping rules where the output is conditional on the time period value.
396 +The main use case is setting the value of Observation [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]] in the target [[dataset>>doc:sdmx:Glossary.Data set.WebHome]].
399 399  
400 -The main use case is setting the value of Observation Attributes in the target dataset.
401 -
402 -|Rule|Source|Target
403 -|1|(((
398 +(% style="width:709.294px" %)
399 +|(% style="width:103px" %)**Rule**|(% style="width:405px" %)**Source**|(% style="width:198px" %)**Target**
400 +|(% style="width:103px" %)1|(% style="width:405px" %)(((
404 404  If
405 -
406 406  INDICATOR=XULADS; and TIME_PERIOD=2007.
407 -)))|(((
408 -Set
409 -
410 -OBS_CONF=F
403 +)))|(% style="width:198px" %)(((
404 +Set OBS_CONF=F
411 411  )))
412 -|2|(((
406 +|(% style="width:103px" %)2|(% style="width:405px" %)(((
413 413  If
414 -
415 415  INDICATOR=XULADS; and TIME_PERIOD=2008.
416 -)))|Set OBS_CONF=F
417 -|3|(((
409 +)))|(% style="width:198px" %)Set OBS_CONF=F
410 +|(% style="width:103px" %)3|(% style="width:405px" %)(((
418 418  If
419 -
420 420  INDICATOR=XULADS; and TIME_PERIOD=2009.
421 -)))|(((
422 -Set
423 -
424 -OBS_CONF=F
413 +)))|(% style="width:198px" %)(((
414 +Set OBS_CONF=F
425 425  )))
426 -|4|(((
416 +|(% style="width:103px" %)4|(% style="width:405px" %)(((
427 427  If
428 -
429 429  INDICATOR=XULADS; and TIME_PERIOD=2010.
430 -)))|(((
431 -Set
432 -
433 -OBS_CONF=**C**
419 +)))|(% style="width:198px" %)(((
420 +Set OBS_CONF=**C**
434 434  )))
435 435  
436 -In the example above, OBS_CONF is an Observation Attribute.
423 +In the example above, OBS_CONF is an Observation [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]].
437 437  
438 -== 13. Time span mapping rules using validity periods ==
425 +== 13.9 Time span mapping rules using validity periods ==
439 439  
440 440  Creating discrete mapping rules for each TIME_PERIOD is impractical where rules need to cover a specific span of time regardless of frequency, and for high-frequency data.
441 441  
... ... @@ -443,25 +443,22 @@
443 443  
444 444  By specifying validity periods, the example from Section 13.8 can be re-written using two rules as follows:
445 445  
446 -|Rule|Source|Target
447 -|1|(((
433 +(% style="width:516.294px" %)
434 +|(% style="width:74px" %)Rule|(% style="width:215px" %)Source|(% style="width:223px" %)Target
435 +|(% style="width:74px" %)1|(% style="width:215px" %)(((
448 448  If
449 -
450 450  INDICATOR=XULADS.
451 -
452 -
453 -Validity Period start period=2007 end period=2009
454 -)))|Set OBS_CONF=F
455 -|2|(((
438 +Validity Period
439 +start period=2007
440 +end period=2009
441 +)))|(% style="width:223px" %)Set OBS_CONF=F
442 +|(% style="width:74px" %)2|(% style="width:215px" %)(((
456 456  If
457 -
458 458  INDICATOR=XULADS.
459 -
460 -
461 -Validity Period start period=2010
462 -)))|(((
445 +Validity Period
446 +start period=2010
447 +)))|(% style="width:223px" %)(((
463 463  Set
464 -
465 465  OBS_CONF=F
466 466  )))
467 467  
... ... @@ -469,86 +469,77 @@
469 469  
470 470  In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
471 471  
472 -== 13. Mapping examples ==
456 +In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C.
473 473  
474 -=== 13. Many to one mapping (N-1) ===
458 +== 13.10 Mapping examples ==
475 475  
476 -|Source|Map To
477 -|(((
478 -**FREQ**="A"
460 +=== 13.10.1 Many to one mapping (N3513 -1) ===
479 479  
462 +(% style="width:542.294px" %)
463 +|(% style="width:274px" %)**Source**|(% style="width:265px" %)**Map To**
464 +|(% style="width:274px" %)(((
465 +(% style="color:#8e44ad" %)**FREQ**="A"
480 480  ADJUSTMENT="N"
481 -
482 482  **REF_AREA**="PL"
483 -
484 484  **COUNTERPART_AREA**="W0"
485 -
486 486  REF_SECTOR="S1"
487 -
488 488  COUNTERPART_SECTOR="S1"
489 -
490 490  ACCOUNTING_ENTRY="B"
491 -
492 492  STO="B5G"
493 -)))|(((
494 -FREQ="A"
495 -
473 +)))|(% style="width:265px" %)(((
474 +(% style="color:#8e44ad" %)FREQ="A"
496 496  REF_AREA="PL"
497 -
498 498  COUNTERPART_AREA="W0"
499 -INDICATOR="IND_ABC"
477 +INDICATOR="IND_ABC"(%%)
500 500  
501 501  )))
502 502  
503 -The bold Dimensions map from source to target verbatim. The mapping simply specifies:
481 +The bold [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] (% style="color:#e74c3c" %)map(%%) from source to target verbatim. The mapping simply specifies:
504 504  
505 -FREQ => FREQ
483 +> FREQ => FREQ
484 +> REF_AREA=> REF_AREA
485 +> COUNTERPART_AREA=> COUNTERPART _AREA
506 506  
507 -REF_AREA=> REF_AREA
487 +No [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required. The source value simply copies across unmodified.
508 508  
509 -COUNTERPART_AREA=> COUNTERPART _AREA
489 +The remaining [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] all (% style="color:#e74c3c" %)map(%%) to the Indicator [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. This is an example of many [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] mapping to one [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. In this case a [[Representation>>doc:sdmx:Glossary.Representation.WebHome]] Mapping is required, and the mapping first describes the input 'partial key' and how this (% style="color:#e74c3c" %)maps(%%) to the target indicator:
510 510  
511 -No Representation Mapping is required. The source value simply copies across unmodified.
491 +> N:S1:S1:B:B5G => IND_ABC
512 512  
513 -The remaining Dimensions all map to the Indicator Dimension. This is an example of many Dimensions mapping to one Dimension. In this case a Representation Mapping is required, and the mapping first describes the input 'partial key' and how this maps to the target indicator:
493 +Where the key sequence is based on the order specified in the (% style="color:#e74c3c" %)mapping(%%) (i.e [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]], REF_SECTOR, etc will result in the first value N being taken from [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]] as this was the first item in the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] list.
514 514  
515 -N:S1:S1:B:B5G => IND_ABC
495 +**Note**: The key order is NOT based on the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] order of the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], as the (% style="color:#e74c3c" %)mapping(%%) needs to be resilient to the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] changing.
516 516  
517 -Where the key sequence is based on the order specified in the mapping (i.e ADJUSTMENT, REF_SECTOR, etc will result in the first value N being taken from ADJUSTMENT as this was the first item in the source Dimension list.
497 +=== 13.10.2 Mapping other data types to Code Id ===
518 518  
519 -**Note**: The key order is NOT based on the Dimension order of the DSD, as the mapping needs to be resilient to the DSD changing.
499 +In the case where the incoming data type is not a string and not a [[code>>doc:sdmx:Glossary.Code.WebHome]] identifier i.e. the source [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
520 520  
521 -1.
522 -11.
523 -111. Mapping other data types to Code Id
501 +The following [[representation>>doc:sdmx:Glossary.Representation.WebHome]] (% style="color:#e74c3c" %)mapping(%%) can be used to explicitly (% style="color:#e74c3c" %)map(%%) each [[age>>doc:sdmx:Glossary.Age.WebHome]] to an output [[code>>doc:sdmx:Glossary.Code.WebHome]].
524 524  
525 -In the case where the incoming data type is not a string and not a code identifier i.e. the source Dimension is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
503 +(% style="width:402.294px" %)
504 +|(% style="width:197px" %)**Source Input Free Text**|(% style="width:204px" %)**Desired Output Code Id**
505 +|(% style="width:197px" %)0|(% style="width:204px" %)A
506 +|(% style="width:197px" %)1|(% style="width:204px" %)A
507 +|(% style="width:197px" %)2|(% style="width:204px" %)A
508 +|(% style="width:197px" %)3|(% style="width:204px" %)B
509 +|(% style="width:197px" %)4|(% style="width:204px" %)B
526 526  
527 -The following representation mapping can be used to explicitly map each age to an output code.
528 -
529 -|Source Input Free Text|Desired Output Code Id
530 -|0|A
531 -|1|A
532 -|2|A
533 -|3|B
534 -|4|B
535 -
536 536  If this mapping takes advantage of regular expressions it can be expressed in two rules:
537 537  
513 +(% style="width:336.294px" %)
514 +|(% style="width:182px" %)**Regular Expression**|(% style="width:151px" %)**Desired Output**
515 +|(% style="width:182px" %)[0-2]|(% style="width:151px" %)A
516 +|(% style="width:182px" %)[3-4]|(% style="width:151px" %)B
538 538  
539 -Regular Expression Desired Output
540 -
541 -|[0-2]|A
542 -|[3-4]|B
543 -
544 544  === 13. Observation Attributes for Time Period ===
545 545  
546 -This use case is where a specific observation for a specific time period has an attribute value.
520 +This use case is where a specific observation for a specific [[time period>>doc:sdmx:Glossary.Time period.WebHome]] has an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] value.
547 547  
548 -|Input INDICATOR|Input TIME_PERIOD|Output OBS_CONF
549 -|XULADS|2008|C
550 -|XULADS|2009|C
551 -|XULADS|2010|C
522 +(% style="width:621.294px" %)
523 +|(% style="width:201px" %)Input INDICATOR|(% style="width:192px" %)Input TIME_PERIOD|(% style="width:225px" %)Output OBS_CONF
524 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2008|(% style="width:225px" %)C
525 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2009|(% style="width:225px" %)C
526 +|(% style="width:201px" %)XULADS|(% style="width:192px" %)2010|(% style="width:225px" %)C
552 552  
553 553  Or using a validity period on the Representation Mapping:
554 554  
1750074994887-415.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +12.4 KB
Content