Changes for page 13 Structure Mapping
Last modified by Artur on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 0 added, 2 removed)
Details
- Page properties
-
- Content
-
... ... @@ -236,22 +236,21 @@ 236 236 |(% style="width:169px" %)ABC_DEF_XYZ|(% style="width:147px" %)5|(% style="width:133px" %)3|(% style="width:171px" %)DEF 237 237 |(% style="width:169px" %)XULADS|(% style="width:147px" %)1|(% style="width:133px" %)2|(% style="width:171px" %)XU 238 238 239 -Sub-strings can therefore be used for the conceptual rule //If starts with 'XU' (% style="color:#e74c3c" %)map(%%)to Y// as shown in the following example:239 +Sub-strings can therefore be used for the conceptual rule //If starts with 'XU' map to Y// as shown in the following example: 240 240 241 241 (% style="width:628.294px" %) 242 242 |(% style="width:163px" %)Start|(% style="width:158px" %)Length|(% style="width:128px" %)Source|(% style="width:176px" %)Target 243 243 |(% style="width:163px" %)1|(% style="width:158px" %)2|(% style="width:128px" %)XU|(% style="width:176px" %)Y 244 244 245 -== 13. 7Mapping non-SDMX time formats to SDMX formats ==245 +== 13.6 Mapping non-SDMX time formats to SDMX formats == 246 246 247 -Structure mapping allows non- [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] compliant time values in source[[datasets>>doc:sdmx:Glossary.Dataset.WebHome]]tobe(% style="color:#e74c3c" %)mapped(%%)to an[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] compliant time format.247 +Structure mapping allows non-SDMX compliant time values in source datasets to be mapped to an SDMX compliant time format. 248 248 249 249 Two types of time input are defined: 250 250 251 -a. **Pattern based dates** – a string which can be described using a notation like dd/mm/yyyy or is represented as the number of periods since a point in time, for example: 2010M001 (first month in 2010), or 2014D123 (123^^rd^^ day in 2014); and 252 -b. **Numerical based datetime** – a number specifying the elapsed periods since a fixed point in time, for example Unix Time is measured by the number of milliseconds since 1970. 251 +a. **Pattern based dates** – a string which can be described using a notation like dd/mm/yyyy or is represented as the number of periods since a point in time, for example: 2010M001 (first month in 2010), or 2014D123 (123^^rd^^ day in 2014); and b. **Numerical based datetime** – a number specifying the elapsed periods since a fixed point in time, for example Unix Time is measured by the number of milliseconds since 1970. 253 253 254 -The output of a time-based mapping is derived from the output Frequency, which is either explicitly stated in the mapping or defined as the value output by a specific [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]or[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]in the output mapping. If the output frequency is unknown or if the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]format is not desired, then additional rules can be provided to specify the output date format for the given frequency Id. The default rules are:253 +The output of a time-based mapping is derived from the output Frequency, which is either explicitly stated in the mapping or defined as the value output by a specific Dimension or Attribute in the output mapping. If the output frequency is unknown or if the SDMX format is not desired, then additional rules can be provided to specify the output date format for the given frequency Id. The default rules are: 255 255 256 256 |Frequency|Format|Example 257 257 |A|YYYY|2010 ... ... @@ -271,95 +271,95 @@ 271 271 272 272 There are two important points to note: 273 273 274 -1. The output frequency determines the output date format, but the default output can be redefined using a Frequency Format mapping to force explicit rules on how the output [[time period>>doc:sdmx:Glossary.Time period.WebHome]] is formatted. 275 -1. To support the use case of changing frequency the structure (% style="color:#e74c3c" %)map(%%) can optionally provide a start of year [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]], which defines the year start date in MM-DD format. For example: YearStart=04-01. 273 +1. The output frequency determines the output date format, but the default output can be redefined using a Frequency Format mapping to force explicit rules on how the output time period is formatted. 274 +1. To support the use case of changing frequency the structure map can optionally provide a start of year attribute, which defines the year start date in MM-DD format. For example: YearStart=04-01. 275 +11. 276 +111. Pattern based dates 276 276 277 - ===13.7.1Pattern based dates===278 +Date and time formats are specified by date and time pattern strings based on Java's Simple Date Format. Within date and time pattern strings, unquoted letters from 'A' to 'Z' and from 'a' to 'z' are interpreted as pattern letters representing the components of a date or time string. Text can be quoted using single quotes (') to avoid interpretation. "''" represents a single quote. All other characters are not interpreted; they're simply copied into the output string during formatting or matched against the input string during parsing. 278 278 279 -Date and [[time formats>>doc:sdmx:Glossary.Time format.WebHome]] are specified by date and time pattern strings based on Java's Simple Date Format. Within date and time pattern strings, unquoted letters from 'A' to 'Z' and from 'a' to 'z' are interpreted as pattern letters representing the [[components>>doc:sdmx:Glossary.Component.WebHome]] of a date or time string. Text can be quoted using single quotes (') to avoid interpretation. "''" represents a single quote. All other characters are not interpreted; they're simply copied into the output string during formatting or matched against the input string during parsing. 280 - 281 281 Due to the fact that dates may differ per locale, an optional property, defining the locale of the pattern, is provided. This would assist processing of source dates, according to the given locale{{footnote}} A list of commonly used locales can be found in the Java supported locales: https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html{{/footnote}}. An indicative list of examples is presented in the following table: 282 282 283 -(% style="width:604.294px" %) 284 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)Australia (AU)|(% style="width:213px" %)en-AU 285 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)Canada (CA)|(% style="width:213px" %)en-CA 286 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)United Kingdom (GB)|(% style="width:213px" %)en-GB 287 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)United States (US)|(% style="width:213px" %)en-US 288 -|(% style="width:172px" %)Estonian (et)|(% style="width:216px" %)Estonia (EE)|(% style="width:213px" %)et-EE 289 -|(% style="width:172px" %)Finnish (fi)|(% style="width:216px" %)Finland (FI)|(% style="width:213px" %)fi-FI 290 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Belgium (BE)|(% style="width:213px" %)fr-BE 291 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Canada (CA)|(% style="width:213px" %)fr-CA 292 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)France (FR)|(% style="width:213px" %)fr-FR 293 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Luxembourg (LU)|(% style="width:213px" %)fr-LU 294 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Switzerland (CH)|(% style="width:213px" %)fr-CH 295 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Austria (AT)|(% style="width:213px" %)de-AT 296 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Germany (DE)|(% style="width:213px" %)de-DE 297 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Luxembourg (LU)|(% style="width:213px" %)de-LU 298 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Switzerland (CH)|(% style="width:213px" %)de-CH 299 -|(% style="width:172px" %)Greek (el)|(% style="width:216px" %)Cyprus (CY)|(% style="width:213px" %)el-CY(*) 300 -|(% style="width:172px" %)Greek (el)|(% style="width:216px" %)Greece (GR)|(% style="width:213px" %)el-GR 301 -|(% style="width:172px" %)Hebrew (iw)|(% style="width:216px" %)Israel (IL)|(% style="width:213px" %)iw-IL 302 -|(% style="width:172px" %)Hindi (hi)|(% style="width:216px" %)India (IN)|(% style="width:213px" %)hi-IN 303 -|(% style="width:172px" %)Hungarian (hu)|(% style="width:216px" %)Hungary (HU)|(% style="width:213px" %)hu-HU 304 -|(% style="width:172px" %)Icelandic (is)|(% style="width:216px" %)Iceland (IS)|(% style="width:213px" %)is-IS 305 -|(% style="width:172px" %)Indonesian (in)|(% style="width:216px" %)Indonesia (ID)|(% style="width:213px" %)in-ID(*) 306 -|(% style="width:172px" %)Irish (ga)|(% style="width:216px" %)Ireland (IE)|(% style="width:213px" %)ga-IE(*) 307 -|(% style="width:172px" %)Italian (it)|(% style="width:216px" %)Italy (IT)|(% style="width:213px" %)it-IT 282 +|English (en)|Australia (AU)|en-AU 283 +|English (en)|Canada (CA)|en-CA 284 +|English (en)|United Kingdom (GB)|en-GB 285 +|English (en)|United States (US)|en-US 286 +|Estonian (et)|Estonia (EE)|et-EE 287 +|Finnish (fi)|Finland (FI)|fi-FI 288 +|French (fr)|Belgium (BE)|fr-BE 289 +|French (fr)|Canada (CA)|fr-CA 290 +|French (fr)|France (FR)|fr-FR 291 +|French (fr)|Luxembourg (LU)|fr-LU 292 +|French (fr)|Switzerland (CH)|fr-CH 293 +|German (de)|Austria (AT)|de-AT 294 +|German (de)|Germany (DE)|de-DE 295 +|German (de)|Luxembourg (LU)|de-LU 296 +|German (de)|Switzerland (CH)|de-CH 297 +|Greek (el)|Cyprus (CY)|el-CY[[(*)>>url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]][[url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 298 +|Greek (el)|Greece (GR)|el-GR 299 +|Hebrew (iw)|Israel (IL)|iw-IL 300 +|Hindi (hi)|India (IN)|hi-IN 301 +|Hungarian (hu)|Hungary (HU)|hu-HU 302 +|Icelandic (is)|Iceland (IS)|is-IS 303 +|Indonesian (in)|Indonesia (ID)|in-ID[[(*)>>url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]][[url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 304 +|Irish (ga)|Ireland (IE)|ga-IE[[(*)>>url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]][[url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 305 +|Italian (it)|Italy (IT)|it-IT 308 308 309 -~* - [[https:~~/~~/www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale>>https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 310 - 311 311 Examples 312 312 313 313 22/06/1981 would be described as dd/MM/YYYY, with locale en-GB 310 + 314 314 2008-mars-12 would be described as YYYY-MMM-DD, with locale fr-FR 312 + 315 315 22 July 1981 would be described as dd MMMM YYYY, with locale en-US 314 + 316 316 22 Jul 1981 would be described as dd MMM YYYY 316 + 317 317 2010 D62 would be described as YYYYDnn (day 62 of the year 2010) 318 318 319 319 The following pattern letters are defined (all other characters from 'A' to 'Z' and from 'a' to 'z' are reserved): 320 320 321 -(% style="width:850.294px" %) 322 -|(% style="width:125px" %)**Letter**|(% style="width:385px" %)**Date or Time Component**|(% style="width:180px" %)**Presentation**|(% style="width:157px" %)**Examples** 323 -|(% style="width:125px" %)G|(% style="width:385px" %)Era designator|(% style="width:180px" %)Text|(% style="width:157px" %)AD 324 -|(% style="width:125px" %)yy|(% style="width:385px" %)Year short (upper case is Year of Week{{footnote}}yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years{{/footnote}})|(% style="width:180px" %)Year|(% style="width:157px" %)96 325 -|(% style="width:125px" %)yyyy|(% style="width:385px" %)Year Full (upper case is Year of Week)|(% style="width:180px" %)Year|(% style="width:157px" %)1996 326 -|(% style="width:125px" %)MM|(% style="width:385px" %)Month number in year starting with 1|(% style="width:180px" %)Month|(% style="width:157px" %)07 327 -|(% style="width:125px" %)MMM|(% style="width:385px" %)Month name short|(% style="width:180px" %)Month|(% style="width:157px" %)Jul 328 -|(% style="width:125px" %)MMMM|(% style="width:385px" %)Month name full|(% style="width:180px" %)Month|(% style="width:157px" %)July 329 -|(% style="width:125px" %)ww|(% style="width:385px" %)Week in year|(% style="width:180px" %)Number|(% style="width:157px" %)27 330 -|(% style="width:125px" %)W|(% style="width:385px" %)Week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2 331 -|(% style="width:125px" %)DD|(% style="width:385px" %)Day in year|(% style="width:180px" %)Number|(% style="width:157px" %)189 332 -|(% style="width:125px" %)dd|(% style="width:385px" %)Day in month|(% style="width:180px" %)Number|(% style="width:157px" %)10 333 -|(% style="width:125px" %)F|(% style="width:385px" %)Day of week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2 334 -|(% style="width:125px" %)E|(% style="width:385px" %)Day name in week|(% style="width:180px" %)Text|(% style="width:157px" %)Tuesday; Tue 335 -|(% style="width:125px" %)U|(% style="width:385px" %)Day number of week (1 = Monday, ..., 7 = Sunday)|(% style="width:180px" %)Number|(% style="width:157px" %)1 336 -|(% style="width:125px" %)HH|(% style="width:385px" %)Hour in day (0-23)|(% style="width:180px" %)Number|(% style="width:157px" %)0 337 -|(% style="width:125px" %)kk|(% style="width:385px" %)Hour in day (1-24)|(% style="width:180px" %)Number|(% style="width:157px" %)24 338 -|(% style="width:125px" %)KK|(% style="width:385px" %)Hour in am/pm (0-11)|(% style="width:180px" %)Number|(% style="width:157px" %)0 339 -|(% style="width:125px" %)hh|(% style="width:385px" %)Hour in am/pm (1-12)|(% style="width:180px" %)Number|(% style="width:157px" %)12 340 -|(% style="width:125px" %)mm|(% style="width:385px" %)Minute in hour|(% style="width:180px" %)Number|(% style="width:157px" %)30 341 -|(% style="width:125px" %)ss|(% style="width:385px" %)Second in minute|(% style="width:180px" %)Number|(% style="width:157px" %)55 342 -|(% style="width:125px" %)S|(% style="width:385px" %)Millisecond|(% style="width:180px" %)Number|(% style="width:157px" %)978 343 -|(% style="width:125px" %)n|(% style="width:385px" %)((( 321 +|Letter|Date or Time Component|Presentation|Examples 322 +|G|Era designator|[[Text>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]]|AD 323 +|yy|Year short (upper case is Year of Week{{footnote}}yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years{{/footnote}})|[[Year>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#year]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#year]]|96 324 +|yyyy|Year Full (upper case is Year of Week)|Year|1996 325 +|MM|Month number in year starting with 1|Month|07 326 +|MMM|Month name short|Month|Jul 327 +|MMMM|Month name full|Month|July 328 +|ww|Week in year|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|27 329 +|W|Week in month|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|2 330 +|DD|Day in year|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|189 331 +|dd|Day in month|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|10 332 +|F|Day of week in month|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|2 333 +|E|Day name in week|[[Text>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]]|Tuesday; Tue 334 +|U|Day number of week (1 = Monday, ..., 7 = Sunday)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|1 335 +|HH|Hour in day (0-23)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|0 336 +|kk|Hour in day (1-24)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|24 337 +|KK|Hour in am/pm (0-11)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|0 338 +|hh|Hour in am/pm (1-12)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|12 339 +|mm|Minute in hour|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|30 340 +|ss|Second in minute|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|55 341 +|S|Millisecond|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|978 342 +|n|((( 344 344 Number of periods, used after a SDMX 344 + 345 345 Frequency Identifier such as M, Q, D (month, quarter, day) 346 -)))| (% style="width:180px" %)Number|(% style="width:157px" %)12346 +)))|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|12 347 347 348 348 The model is illustrated below: 349 349 350 -[[image:1750074822764-573.png]] 351 351 351 + 352 352 **Figure 24 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD dimension with the additional information on the component map to describe the time format?** 353 353 354 -[[image:1750074865924-797.png]] 355 355 356 -(% class="wikigeneratedid" id="HFigure25showinganinputdateformat2CwhoseoutputfrequencyisderivedfromtheoutputvalueoftheFREQDimension" %) 357 -**Figure 25 showing an input date format, whose output frequency is derived from the output value of the FREQ Dimension** 358 358 359 -=== 13.7.2Numericalbaseddatetime ===356 +==== Figure 25 showing an input date format, whose output frequency is derived from the output value of the FREQ Dimension ==== 360 360 361 - Wherethesource datetimeinput is purely numerical,the mapping rules are definedby the **Base** as a valid[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[TimePeriod>>doc:sdmx:Glossary.Time period.WebHome]], and the **Period** which must take one of the following enumerated values:358 +=== 13.3.6 Numerical based datetime === 362 362 360 +Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid SDMX Time Period, and the **Period** which must take one of the following enumerated values: 361 + 363 363 * day 364 364 * second 365 365 * millisecond ... ... @@ -366,21 +366,23 @@ 366 366 * microsecond 367 367 * nanosecond 368 368 369 -(% style="width:545.294px" %) 370 -|(% style="width:268px" %)Numerical datetime systems|(% style="width:121px" %)Base|(% style="width:153px" %)Period 371 -|(% style="width:268px" %)((( 368 +|Numerical datetime systems|Base|Period 369 +|((( 372 372 Epoch Time (UNIX) 371 + 373 373 Milliseconds since 01 Jan 1970 374 -)))| (% style="width:121px" %)1970|(% style="width:153px" %)millisecond375 -|( % style="width:268px" %)(((373 +)))|1970|millisecond 374 +|((( 376 376 Windows System Time 376 + 377 377 Milliseconds since 01 Jan 1601 378 -)))| (% style="width:121px" %)1601|(% style="width:153px" %)millisecond378 +)))|1601|millisecond 379 379 380 380 The example above illustrates numerical based datetime mapping rules for two commonly used time standards. 381 381 382 382 The model is illustrated below: 383 383 384 +[[image:1750072341491-790.jpeg]] 384 384 385 385 **Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the** 386 386 ... ... @@ -515,8 +515,8 @@ 515 515 516 516 **Note**: The key order is NOT based on the Dimension order of the DSD, as the mapping needs to be resilient to the DSD changing. 517 517 518 -1. 519 -11. 519 +1. 520 +11. 520 520 111. Mapping other data types to Code Id 521 521 522 522 In the case where the incoming data type is not a string and not a code identifier i.e. the source Dimension is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions.
- 1750074822764-573.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -25.1 KB - Content
- 1750074865924-797.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -46.8 KB - Content