Changes for page 13 Structure Mapping
Last modified by Artur on 2025/09/10 11:19
Summary
-
Page properties (1 modified, 0 added, 0 removed)
-
Attachments (0 modified, 0 added, 3 removed)
Details
- Page properties
-
- Content
-
... ... @@ -1,52 +1,7 @@ 1 -(% contenteditable="false" tabindex="-1" %) 2 -((( 3 -(% class="macro" data-macro="startmacro:box|-|title=~"**Contents**~"|-|\{\{toc/}}" data-widget="xwiki-macro" %) 4 -((( 5 -(% class="macro-placeholder hidden" %) 6 -((( 7 -macro:box 8 -))) 1 +{{box title="**Contents**"}} 2 +{{toc/}} 3 +{{/box}} 9 9 10 -(% class="box" %) 11 -((( 12 -(% class="box-title" %) 13 -((( 14 -**Contents** 15 -))) 16 - 17 -(% class="macro" data-macro="startmacro:toc|-|" %) 18 -((( 19 -(% class="macro-placeholder hidden" %) 20 -((( 21 -macro:toc 22 -))) 23 - 24 -(% class="wikitoc" %) 25 -* 26 -** [[13.1 Introduction>>doc:null||anchor="H13.1Introduction"]] 27 -** [[13.2 1-1 structure maps>>doc:null||anchor="H13.21-1structuremaps"]] 28 -** [[13.3 N-n structure maps>>doc:null||anchor="H13.3N-nstructuremaps"]] 29 -** [[13.4 Ambiguous mapping rules>>doc:null||anchor="H13.4Ambiguousmappingrules"]] 30 -** [[13.5 Representation maps>>doc:null||anchor="H13.5Representationmaps"]] 31 -** [[13.6 Regular expression and substring rules>>doc:null||anchor="H13.6Regularexpressionandsubstringrules"]] 32 -*** [[13.6.1 Regular expressions>>doc:null||anchor="H13.6.1Regularexpressions"]] 33 -*** [[13.6.2 Substrings>>doc:null||anchor="H13.6.2Substrings"]] 34 -** [[13.7 Mapping non-SDMX time formats to SDMX formats>>doc:null||anchor="H13.7Mappingnon-SDMXtimeformatstoSDMXformats"]] 35 -*** [[13.7.1 Pattern based dates>>doc:null||anchor="H13.7.1Patternbaseddates"]] 36 -*** [[13.7.2 Numerical based datetime>>doc:null||anchor="H13.7.2Numericalbaseddatetime"]] 37 -*** [[13.7.3 Mapping more complex time inputs>>doc:null||anchor="H13.7.3Mappingmorecomplextimeinputs"]] 38 -** [[13.8 Using TIME_PERIOD in mapping rules>>doc:null||anchor="H13.8UsingTIME_PERIODinmappingrules"]] 39 -** [[13.9 Time span mapping rules using validity periods>>doc:null||anchor="H13.9Timespanmappingrulesusingvalidityperiods"]] 40 -** [[13.10 Mapping examples>>doc:null||anchor="H13.10Mappingexamples"]] 41 -*** [[13.10.1 Many to one mapping (N3513 -1)>>doc:null||anchor="H13.10.1A0Manytoonemapping28N3513-129"]] 42 -*** [[13.10.2 Mapping other data types to Code Id>>doc:null||anchor="H13.10.2MappingotherdatatypestoCodeId"]] 43 -*** [[13.10.3 Observation Attributes for Time Period>>doc:null||anchor="H13.10.3ObservationAttributesforTimePeriod"]] 44 -*** [[13.10.4 Time mapping>>doc:null||anchor="H13.10.4Timemapping"]] 45 -))) 46 -))) 47 -))) 48 -))) 49 - 50 50 == 13.1 Introduction == 51 51 52 52 The purpose of [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] structure mapping is to transform [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] from one dimensionality to another. In practice, this means that the input and output [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] conform to different Data Structure Definition. ... ... @@ -63,7 +63,7 @@ 63 63 64 64 * Transforming received data into a common internal structure; 65 65 * Transforming reported data into the data collector's preferred structure; 66 -* Transforming unidimensional [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] (% contenteditable="false" tabindex="-1" data-macro="startmacro:footnote|-||-|Unidimensional datasets are those with a single 'indicator' or 'series code' dimension." data-widget="xwiki-macro" class="macro hidden macro-placeholder" %)macro:footnote(%contenteditable="false" tabindex="-1" data-macro="startmacro:footnote|-||-|Unidimensionaldatasets are those with a single 'indicator' or 'series code' dimension." data-widget="xwiki-macro" class="macro footnoteRef" id="x_footnote_ref_1" %)^^[[1>>doc:null||anchor="x_footnote_1"]]^^(%%) to multi-dimensional; and21 +* Transforming unidimensional [[datasets>>doc:sdmx:Glossary.Data set.WebHome]]{{footnote}}Unidimensional datasets are those with a single 'indicator' or 'series code' dimension.{{/footnote}} to multi-dimensional; and 67 67 * Transforming internal [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] with a complex structure to a simpler structure with fewer [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] suitable for dissemination. 68 68 69 69 == 13.2 1-1 structure maps == ... ... @@ -81,62 +81,66 @@ 81 81 |(% style="width:173px" %)Andorra|(% style="width:180px" %)AD|(% style="width:229px" %)AND 82 82 |(% style="width:173px" %)etc…|(% style="width:180px" %) |(% style="width:229px" %) 83 83 84 -Different source values can also (% style="color:#e74c3c" %)map(%%)to the same target value, for example when deriving regions from country[[codes>>doc:sdmx:Glossary.Code.WebHome]].39 +Different source values can also map to the same target value, for example when deriving regions from country codes. 85 85 86 -(% style="width:490.294px" %) 87 -|(% style="width:260px" %)Source Component: REF_AREA|(% style="width:227px" %)Target Component: REGION 88 -|(% style="width:260px" %)FR|(% style="width:227px" %)EUR 89 -|(% style="width:260px" %)DE|(% style="width:227px" %)EUR 90 -|(% style="width:260px" %)IT|(% style="width:227px" %)EUR 91 -|(% style="width:260px" %)ES|(% style="width:227px" %)EUR 92 -|(% style="width:260px" %)BE|(% style="width:227px" %)EUR 41 +|Source Component: REF_AREA|Target Component: REGION 42 +|FR|EUR 43 +|DE|EUR 44 +|IT|EUR 45 +|ES|EUR 46 +|BE|EUR 93 93 94 94 == 13.3 N-n structure maps == 95 95 96 -N-n (pronounced 'N to N') mappings describe rules where a specified combination of values in multiple source [[Components>>doc:sdmx:Glossary.Component.WebHome]](% style="color:#e74c3c" %)map(%%)to specified values in one or more target[[Components>>doc:sdmx:Glossary.Component.WebHome]].For example, when mapping a partial[[Series Key>>doc:sdmx:Glossary.Serieskey.WebHome]]from a highly multidimensional cube (like Balance of Payments) to a single 'Indicator'[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]in a target Data Structure.50 +N-n (pronounced 'N to N') mappings describe rules where a specified combination of values in multiple source Components map to specified values in one or more target Components. For example, when mapping a partial Series Key from a highly multidimensional cube (like Balance of Payments) to a single 'Indicator' Dimension in a target Data Structure. 97 97 98 98 Example: 99 99 100 -(% style="width:964.294px" %) 101 -|(% style="width:65px" %)Rule|(% style="width:519px" %)Source|(% style="width:378px" %)Target 102 -|(% style="width:65px" %)1|(% style="width:519px" %)((( 54 +|Rule|Source|Target 55 +|1|((( 103 103 If 104 -FREQUENCY=A; and 105 -ADJUSTMENT=N; and 106 -MATURITY=L. 107 -)))|(% style="width:378px" %)((( 57 + 58 +FREQUENCY=A; and ADJUSTMENT=N; and MATURITY=L. 59 +)))|((( 108 108 Set 61 + 109 109 INDICATOR=A_N_L 110 110 ))) 111 -| (% style="width:65px" %)2|(% style="width:519px" %)(((64 +|2|((( 112 112 If 113 -FREQUENCY=M; and 114 -ADJUSTMENT=S_A1; and 115 -MATURITY=TY12. 116 -)))|(% style="width:378px" %)((( 66 + 67 +FREQUENCY=M; and ADJUSTMENT=S_A1; and MATURITY=TY12. 68 +)))|((( 117 117 Set 70 + 118 118 INDICATOR=MON_SAX_12 119 119 ))) 120 120 121 121 N-n rules can also set values for multiple source Components. 122 122 123 -(% style="width:965.294px" %) 124 -|(% style="width:73px" %)Rule|(% style="width:506px" %)Source|(% style="width:383px" %)Target 125 -|(% style="width:73px" %)1|(% style="width:506px" %)((( 76 +|Rule|Source|Target 77 +|1|((( 126 126 If 79 + 127 127 FREQUENCY=A; and ADJUSTMENT=N; and MATURITY=L. 128 -)))|( % style="width:383px" %)(((81 +)))|((( 129 129 Set 83 + 130 130 INDICATOR=A_N_L, STATUS=QXR15, 85 + 131 131 NOTE="Unadjusted". 132 132 ))) 133 -| (% style="width:73px" %)2|(% style="width:506px" %)(((88 +|2|((( 134 134 If 90 + 135 135 FREQUENCY=M; and ADJUSTMENT=S_A1; and MATURITY=TY12. 136 -)))|( % style="width:383px" %)(((92 +)))|((( 137 137 Set 94 + 138 138 INDICATOR=MON_SAX_12, 96 + 139 139 STATUS=MPM12, 98 + 140 140 NOTE="Seasonally Adjusted" 141 141 ))) 142 142 ... ... @@ -146,157 +146,156 @@ 146 146 147 147 A simple example mapping a source dataset with a single dimension to one with multiple dimensions is shown below: 148 148 149 -(% style="width:972.294px" %) 150 -|(% style="width:257px" %)Source|(% style="width:315px" %)Target|(% style="width:397px" %)Output Series Key 151 -|(% style="width:257px" %)SERIES_CODE=XMAN_Z_21|(% style="width:315px" %)((( 108 +|Source|Target|Output Series Key 109 +|SERIES_CODE=XMAN_Z_21|((( 152 152 Dimensions 111 + 153 153 INDICATOR=XM 154 154 FREQ=A 155 155 ADJUSTMENT=N 156 156 Attributes 157 157 UNIT_MEASURE=_Z 117 + 158 158 COMP_ORG=21 159 -)))| (% style="width:397px" %)XM:A:N160 -|( % style="width:257px" %)(((119 +)))|XM:A:N 120 +|((( 161 161 SERIES_CODE=XMAN_Z_34 162 162 163 163 164 -)))|( % style="width:315px" %)(((124 +)))|((( 165 165 Dimensions 126 + 166 166 INDICATOR=XM 128 + 167 167 FREQ=A 130 + 168 168 ADJUSTMENT=N 132 + 169 169 Attributes 134 + 170 170 UNIT_MEASURE=_Z 136 + 171 171 COMP_ORG=34 172 -)))| (% style="width:397px" %)XM:A:N138 +)))|XM:A:N 173 173 174 -The above behaviour can be okay if the series XMAN_Z_21 contains observations for different periods of time then the series XMAN_Z_34. If however both series contain observations for the same point in time, the output for this mapping will be two observations with the same [[series key>>doc:sdmx:Glossary.Series key.WebHome]], for the same period in time.140 +The above behaviour can be okay if the series XMAN_Z_21 contains observations for different periods of time then the series XMAN_Z_34. If however both series contain observations for the same point in time, the output for this mapping will be two observations with the same series key, for the same period in time. 175 175 176 176 == 13.5 Representation maps == 177 177 178 - [[Representation>>doc:sdmx:Glossary.Representation.WebHome]](% style="color:#e74c3c" %)Maps(%%)replace the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]2.1 Codelist(% style="color:#e74c3c" %)Maps(%%)and are used describe explicit mappings between source and target[[Component>>doc:sdmx:Glossary.Component.WebHome]]values.144 +Representation Maps replace the SDMX 2.1 Codelist Maps and are used describe explicit mappings between source and target Component values. 179 179 180 -The source and target of a [[Representation>>doc:sdmx:Glossary.Representation.WebHome]](% style="color:#e74c3c" %)Map(%%)can reference any of the following:146 +The source and target of a Representation Map can reference any of the following: 181 181 182 182 1. Codelist 183 183 1. Free Text (restricted by type, e.g String, Integer, Boolean) 184 184 1. Valuelist 185 185 186 -A [[Representation>>doc:sdmx:Glossary.Representation.WebHome]](% style="color:#e74c3c" %)Map(%%)mapping ISO 2-character to ISO 3-character Codelists would take the following form:152 +A Representation Map mapping ISO 2-character to ISO 3-character Codelists would take the following form: 187 187 188 -(% style="width:356.294px" %) 189 -|(% style="width:167px" %)CL_ISO_ALPHA2|(% style="width:186px" %)CL_ISO_ALPHA3 190 -|(% style="width:167px" %)AF|(% style="width:186px" %)AFG 191 -|(% style="width:167px" %)AL|(% style="width:186px" %)ALB 192 -|(% style="width:167px" %)DZ|(% style="width:186px" %)DZA 193 -|(% style="width:167px" %)AS|(% style="width:186px" %)ASM 194 -|(% style="width:167px" %)AD|(% style="width:186px" %)AND 195 -|(% style="width:167px" %)etc…|(% style="width:186px" %) 154 +|CL_ISO_ALPHA2|CL_ISO_ALPHA3 155 +|AF|AFG 156 +|AL|ALB 157 +|DZ|DZA 158 +|AS|ASM 159 +|AD|AND 160 +|etc…| 196 196 197 -A [[Representation>>doc:sdmx:Glossary.Representation.WebHome]](% style="color:#e74c3c" %)Map(%%)mapping free text country names to an ISO 2-character Codelist could be similarly described:162 +A Representation Map mapping free text country names to an ISO 2-character Codelist could be similarly described: 198 198 199 -(% style="width:364.294px" %) 200 -|(% style="width:169px" %)Text|(% style="width:192px" %)CL_ISO_ALPHA2 201 -|(% style="width:169px" %)"Germany"|(% style="width:192px" %)DE 202 -|(% style="width:169px" %)"France"|(% style="width:192px" %)FR 203 -|(% style="width:169px" %)"United Kingdom"|(% style="width:192px" %)GB 204 -|(% style="width:169px" %)"Great Britain"|(% style="width:192px" %)GB 205 -|(% style="width:169px" %)"Ireland"|(% style="width:192px" %)IE 206 -|(% style="width:169px" %)"Eire"|(% style="width:192px" %)IE 207 -|(% style="width:169px" %)etc…|(% style="width:192px" %) 164 +|Text|CL_ISO_ALPHA2 165 +|"Germany"|DE 166 +|"France"|FR 167 +|"United Kingdom"|GB 168 +|"Great Britain"|GB 169 +|"Ireland"|IE 170 +|"Eire"|IE 171 +|etc…| 208 208 209 -Valuelists, introduced in [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]3.0, are equivalent to Codelists but allow the maintenance of non-[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]identifiers. Importantly, their IDs do not need to conform to IDType, but as a consequence are not Identifiable.173 +Valuelists, introduced in SDMX 3.0, are equivalent to Codelists but allow the maintenance of non-SDMX identifiers. Importantly, their IDs do not need to conform to IDType, but as a consequence are not Identifiable. 210 210 211 -When used in [[Representation>>doc:sdmx:Glossary.Representation.WebHome]](% style="color:#e74c3c" %)Maps(%%), Valuelists allow Non-[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]identifiers containing characters like £, $, % to be(% style="color:#e74c3c" %)mapped(%%)to[[Code>>doc:sdmx:Glossary.Code.WebHome]]IDs, or[[Codes>>doc:sdmx:Glossary.Code.WebHome]](% style="color:#e74c3c" %)mapped(%%)to non-[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]identifiers.175 +When used in Representation Maps, Valuelists allow Non-SDMX identifiers containing characters like £, $, % to be mapped to Code IDs, or Codes mapped to non-SDMX identifiers. 212 212 213 213 In common with Codelists, each item in a Valuelist has a multilingual name giving it a human-readable label and an optional description. For example: 214 214 215 -(% style="width:435.294px" %) 216 -|(% style="width:126px" %)Value|(% style="width:133px" %)Locale|(% style="width:173px" %)Name 217 -|(% style="width:126px" %)$|(% style="width:133px" %)en|(% style="width:173px" %)United States Dollar 218 -|(% style="width:126px" %)%|(% style="width:133px" %)En|(% style="width:173px" %)Percentage 219 -|(% style="width:126px" %) |(% style="width:133px" %)fr|(% style="width:173px" %)Pourcentage 179 +|Value|Locale|Name 180 +|$|en|United States Dollar 181 +|%|En|Percentage 182 +| |fr|Pourcentage 220 220 221 -Other characteristics of [[Representation>>doc:sdmx:Glossary.Representation.WebHome]](% style="color:#e74c3c" %)Maps(%%):184 +Other characteristics of Representation Maps: 222 222 223 -* Support the (% style="color:#e74c3c" %)mapping(%%) of multiple source [[Component>>doc:sdmx:Glossary.Component.WebHome]] values to multiple Target [[Component>>doc:sdmx:Glossary.Component.WebHome]] values as described in section 13.3 on n-to-n mappings; this covers also the case of (% style="color:#e74c3c" %)mapping(%%) an [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]] with an array [[representation>>doc:sdmx:Glossary.Representation.WebHome]] to (% style="color:#e74c3c" %)map(%%) combinations of values to a single target value; 224 -* Allow source or target mappings for an Item to be optional allowing rules such as 'A (% style="color:#e74c3c" %)maps(%%) to nothing' or 'nothing (% style="color:#e74c3c" %)maps(%%) to A'; and 225 -* Support for (% style="color:#e74c3c" %)mapping(%%) rules where regular expressions or substrings are used to match source [[Component>>doc:sdmx:Glossary.Component.WebHome]] values. Refer to section 13.6 for more on this topic. 186 +* Support the mapping of multiple source Component values to multiple Target Component values as described in section 13.3 on n-to-n mappings; this covers also the case of mapping an Attribute with an array representation to map combinations of values to a single target value; 187 +* Allow source or target mappings for an Item to be optional allowing rules such as 'A maps to nothing' or 'nothing maps to A'; and 188 +* Support for mapping rules where regular expressions or substrings are used to match source Component values. Refer to section 13.6 for more on this topic. 189 +*1. Regular expression and substring rules 226 226 227 - ==13.6Regular expressionndsubstringrules==191 +It is common for classifications to contain meanings within the identifier, for example the code Id 'XULADS' may refer to a particular seasonality because it starts with the letters XU. 228 228 229 - Itiscommonforclassificationstocontain meanings withintheidentifier, forexample the[[code>>doc:sdmx:Glossary.Code.WebHome]]Id'XULADS'may refer toparticularseasonalitybecause it startswiththelettersXU.193 +With SDMX 2.1 each code that starts with XU had to be individually mapped to the same seasonality, and additional mappings added when new Codes were added to the Codelists. This led to many hundreds or thousands of mappings which can be more efficiently summarised in a single conceptual rule: 230 230 231 -With [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] 2.1 each [[code>>doc:sdmx:Glossary.Code.WebHome]] that starts with XU had to be individually (% style="color:#e74c3c" %)mapped(%%) to the same seasonality, and additional mappings added when new [[Codes>>doc:sdmx:Glossary.Code.WebHome]] were added to the Codelists. This led to many hundreds or thousands of mappings which can be more efficiently summarised in a single conceptual rule: 232 - 233 233 //If starts with 'XU' map to 'Y'// 234 234 235 235 These rules are described using either regular expressions, or substrings for simpler use cases. 236 236 237 -=== 13. 6.1 Regular expressions ===199 +=== 13.5.1 Regular expressions === 238 238 239 -Regular expression mapping rules are defined in the [[Representation>>doc:sdmx:Glossary.Representation.WebHome]]Map.201 +Regular expression mapping rules are defined in the Representation Map. 240 240 241 -Below is an example set of regular expression rules for a particular [[component>>doc:sdmx:Glossary.Component.WebHome]].203 +Below is an example set of regular expression rules for a particular component. 242 242 243 -(% style="width:664.294px" %) 244 -|(% style="width:141px" %)**Regex**|(% style="width:362px" %)**Description**|(% style="width:158px" %)**Output** 245 -|(% style="width:141px" %)A|(% style="width:362px" %)Rule match if input = 'A'|(% style="width:158px" %)OUT_A 246 -|(% style="width:141px" %)^[A-G]|(% style="width:362px" %)Rule match if the input starts with letters A to G|(% style="width:158px" %)OUT_B 247 -|(% style="width:141px" %)A~|B|(% style="width:362px" %)Rule match if input is either 'A' or 'B'|(% style="width:158px" %)OUT_C 205 +|Regex|Description|Output 206 +|A|Rule match if input = 'A'|OUT_A 207 +|^[A-G]|Rule match if the input starts with letters A to G|OUT_B 208 +|A~|B|Rule match if input is either 'A' or 'B'|OUT_C 248 248 249 -Like all mapping rules, the output is either a [[Code>>doc:sdmx:Glossary.Code.WebHome]], a Value or free text depending on the[[representation>>doc:sdmx:Glossary.Representation.WebHome]]of the[[Component>>doc:sdmx:Glossary.Component.WebHome]]in the target[[Data Structure Definition>>doc:sdmx:Glossary.Data structure definition.WebHome]].210 +Like all mapping rules, the output is either a Code, a Value or free text depending on the representation of the Component in the target Data Structure Definition. 250 250 251 251 If the regular expression contains capture groups, these can be used in the definition of the output value, by specifying \**//n//**// //as an output value where **//n//** is the number of the capture group starting from 1. For example 252 252 253 -(% style="width:700.294px" %) 254 -|(% style="width:203px" %)Regex|(% style="width:148px" %)Target output|(% style="width:157px" %)Example Input|(% style="width:189px" %)Example Output 255 -|(% style="width:203px" %)((( 256 -([0-9]{4})[0-9]([0-9]{1}) 257 -)))|(% style="width:148px" %)\1-Q\2|(% style="width:157px" %)200933|(% style="width:189px" %)2009-Q3 214 +|Regex|Target output|Example Input|Example Output 215 +|((( 216 +([0-9]{4})[0- 258 258 218 +9]([0-9]{1}) 219 +)))|\1-Q\2|200933|2009-Q3 220 + 259 259 As regular expression rules can be used as a general catch-all if nothing else matches, the ordering of the rules is important. Rules should be tested starting with the highest priority, moving down the list until a match is found. 260 260 261 261 The following example shows this: 262 262 263 -(% style="width:704.294px" %) 264 -|(% style="width:130px" %)Priority|(% style="width:125px" %)Regex|(% style="width:241px" %)Description|(% style="width:205px" %)Output 265 -|(% style="width:130px" %)1|(% style="width:125px" %)A|(% style="width:241px" %)Rule match if input = 'A'|(% style="width:205px" %)OUT_A 266 -|(% style="width:130px" %)2|(% style="width:125px" %)B|(% style="width:241px" %)Rule match if input = 'B'|(% style="width:205px" %)OUT_B 267 -|(% style="width:130px" %)3|(% style="width:125px" %)[A-Z]|(% style="width:241px" %)Any character A-Z|(% style="width:205px" %)OUT_C 225 +|Priority|Regex|Description|Output 226 +|1|A|Rule match if input = 'A'|OUT_A 227 +|2|B|Rule match if input = 'B'|OUT_B 228 +|3|[A-Z]|Any character A-Z|OUT_C 268 268 269 269 The input 'A' matches both the first and the last rule, but the first takes precedence having the higher priority. The output is OUT_A. 270 270 271 271 The input 'G' matches on the last rule which is used as a catch-all or default in this example. 272 272 273 -=== 13. 6.2Substrings ===234 +=== 13. Substrings === 274 274 275 275 Substrings provide an alternative to regular expressions where the required section of an input value can be described using the number of the starting character, and the length of the substring in characters. The first character is at position 1. 276 276 277 277 For instance: 278 278 279 -(% style="width:623.294px" %) 280 -|(% style="width:169px" %)Input String|(% style="width:147px" %)Start|(% style="width:133px" %)Length|(% style="width:171px" %)Output 281 -|(% style="width:169px" %)ABC_DEF_XYZ|(% style="width:147px" %)5|(% style="width:133px" %)3|(% style="width:171px" %)DEF 282 -|(% style="width:169px" %)XULADS|(% style="width:147px" %)1|(% style="width:133px" %)2|(% style="width:171px" %)XU 240 +|Input String|Start|Length|Output 241 +|ABC_DEF_XYZ|5|3|DEF 242 +|XULADS|1|2|XU 283 283 284 -Sub-strings can therefore be used for the conceptual rule //If starts with 'XU' (% style="color:#e74c3c" %)map(%%)to Y// as shown in the following example:244 +Sub-strings can therefore be used for the conceptual rule //If starts with 'XU' map to Y// as shown in the following example: 285 285 286 -(% style="width:628.294px" %) 287 -|(% style="width:163px" %)Start|(% style="width:158px" %)Length|(% style="width:128px" %)Source|(% style="width:176px" %)Target 288 -|(% style="width:163px" %)1|(% style="width:158px" %)2|(% style="width:128px" %)XU|(% style="width:176px" %)Y 246 +|Start|Length|Source|Target 247 +|1|2|XU|Y 289 289 290 -== 13. 7Mapping non-SDMX time formats to SDMX formats ==249 +== 13.6 Mapping non-SDMX time formats to SDMX formats == 291 291 292 -Structure mapping allows non- [[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] compliant time values in source[[datasets>>doc:sdmx:Glossary.Dataset.WebHome]]tobe(% style="color:#e74c3c" %)mapped(%%)to an[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]] compliant time format.251 +Structure mapping allows non-SDMX compliant time values in source datasets to be mapped to an SDMX compliant time format. 293 293 294 294 Two types of time input are defined: 295 295 296 -a. **Pattern based dates** – a string which can be described using a notation like dd/mm/yyyy or is represented as the number of periods since a point in time, for example: 2010M001 (first month in 2010), or 2014D123 (123^^rd^^ day in 2014); and 297 -b. **Numerical based datetime** – a number specifying the elapsed periods since a fixed point in time, for example Unix Time is measured by the number of milliseconds since 1970. 255 +a. **Pattern based dates** – a string which can be described using a notation like dd/mm/yyyy or is represented as the number of periods since a point in time, for example: 2010M001 (first month in 2010), or 2014D123 (123^^rd^^ day in 2014); and b. **Numerical based datetime** – a number specifying the elapsed periods since a fixed point in time, for example Unix Time is measured by the number of milliseconds since 1970. 298 298 299 -The output of a time-based mapping is derived from the output Frequency, which is either explicitly stated in the mapping or defined as the value output by a specific [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]or[[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]]in the output mapping. If the output frequency is unknown or if the[[SDMX>>doc:sdmx:Glossary.Statisticaldata and metadata exchange.WebHome]]format is not desired, then additional rules can be provided to specify the output date format for the given frequency Id. The default rules are:257 +The output of a time-based mapping is derived from the output Frequency, which is either explicitly stated in the mapping or defined as the value output by a specific Dimension or Attribute in the output mapping. If the output frequency is unknown or if the SDMX format is not desired, then additional rules can be provided to specify the output date format for the given frequency Id. The default rules are: 300 300 301 301 |Frequency|Format|Example 302 302 |A|YYYY|2010 ... ... @@ -316,95 +316,95 @@ 316 316 317 317 There are two important points to note: 318 318 319 -1. The output frequency determines the output date format, but the default output can be redefined using a Frequency Format mapping to force explicit rules on how the output [[time period>>doc:sdmx:Glossary.Time period.WebHome]] is formatted. 320 -1. To support the use case of changing frequency the structure (% style="color:#e74c3c" %)map(%%) can optionally provide a start of year [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]], which defines the year start date in MM-DD format. For example: YearStart=04-01. 277 +1. The output frequency determines the output date format, but the default output can be redefined using a Frequency Format mapping to force explicit rules on how the output time period is formatted. 278 +1. To support the use case of changing frequency the structure map can optionally provide a start of year attribute, which defines the year start date in MM-DD format. For example: YearStart=04-01. 279 +11. 280 +111. Pattern based dates 321 321 322 - ===13.7.1Pattern based dates===282 +Date and time formats are specified by date and time pattern strings based on Java's Simple Date Format. Within date and time pattern strings, unquoted letters from 'A' to 'Z' and from 'a' to 'z' are interpreted as pattern letters representing the components of a date or time string. Text can be quoted using single quotes (') to avoid interpretation. "''" represents a single quote. All other characters are not interpreted; they're simply copied into the output string during formatting or matched against the input string during parsing. 323 323 324 -D ateand[[time formats>>doc:sdmx:Glossary.Timeformat.WebHome]] arespecifiedby date andtimepattern stringsbasedon Java's SimpleDate Format.Within dateand timepatternstrings,unquotedlettersfrom 'A' to 'Z' andfrom 'a'to'z' areterpretedas pattern letters representing the[[components>>doc:sdmx:Glossary.Component.WebHome]]ofa dateortimestring.Textcan bequotedusingsinglequotes(') toavoidinterpretation."''" representssinglequote.Allothercharacters arenot interpreted; they'resimply copied intoheoutputstringduring formattingormatchedagainstthe inputstringduringparsing.284 +Due to the fact that dates may differ per locale, an optional property, defining the locale of the pattern, is provided. This would assist processing of source dates, according to the given locale{{footnote}} A list of commonly used locales can be found in the Java supported locales: https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html{{/footnote}}. An indicative list of examples is presented in the following table: 325 325 326 -Due to the fact that dates may differ per locale, an optional property, defining the locale of the pattern, is provided. This would assist processing of source dates, according to the given locale(% contenteditable="false" tabindex="-1" data-macro="startmacro:footnote|-||-|A list of commonly used locales can be found in the Java supported locales: https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html" data-widget="xwiki-macro" class="macro hidden macro-placeholder" %)macro:footnote(% contenteditable="false" tabindex="-1" data-macro="startmacro:footnote|-||-|A list of commonly used locales can be found in the Java supported locales: https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html" data-widget="xwiki-macro" class="macro footnoteRef" id="x_footnote_ref_2" %)^^[[2>>doc:null||anchor="x_footnote_2"]]^^(%%). An indicative list of examples is presented in the following table: 286 +|English (en)|Australia (AU)|en-AU 287 +|English (en)|Canada (CA)|en-CA 288 +|English (en)|United Kingdom (GB)|en-GB 289 +|English (en)|United States (US)|en-US 290 +|Estonian (et)|Estonia (EE)|et-EE 291 +|Finnish (fi)|Finland (FI)|fi-FI 292 +|French (fr)|Belgium (BE)|fr-BE 293 +|French (fr)|Canada (CA)|fr-CA 294 +|French (fr)|France (FR)|fr-FR 295 +|French (fr)|Luxembourg (LU)|fr-LU 296 +|French (fr)|Switzerland (CH)|fr-CH 297 +|German (de)|Austria (AT)|de-AT 298 +|German (de)|Germany (DE)|de-DE 299 +|German (de)|Luxembourg (LU)|de-LU 300 +|German (de)|Switzerland (CH)|de-CH 301 +|Greek (el)|Cyprus (CY)|el-CY[[(*)>>url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]][[url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 302 +|Greek (el)|Greece (GR)|el-GR 303 +|Hebrew (iw)|Israel (IL)|iw-IL 304 +|Hindi (hi)|India (IN)|hi-IN 305 +|Hungarian (hu)|Hungary (HU)|hu-HU 306 +|Icelandic (is)|Iceland (IS)|is-IS 307 +|Indonesian (in)|Indonesia (ID)|in-ID[[(*)>>url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]][[url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 308 +|Irish (ga)|Ireland (IE)|ga-IE[[(*)>>url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]][[url:https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 309 +|Italian (it)|Italy (IT)|it-IT 327 327 328 -(% style="width:604.294px" %) 329 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)Australia (AU)|(% style="width:213px" %)en-AU 330 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)Canada (CA)|(% style="width:213px" %)en-CA 331 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)United Kingdom (GB)|(% style="width:213px" %)en-GB 332 -|(% style="width:172px" %)English (en)|(% style="width:216px" %)United States (US)|(% style="width:213px" %)en-US 333 -|(% style="width:172px" %)Estonian (et)|(% style="width:216px" %)Estonia (EE)|(% style="width:213px" %)et-EE 334 -|(% style="width:172px" %)Finnish (fi)|(% style="width:216px" %)Finland (FI)|(% style="width:213px" %)fi-FI 335 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Belgium (BE)|(% style="width:213px" %)fr-BE 336 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Canada (CA)|(% style="width:213px" %)fr-CA 337 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)France (FR)|(% style="width:213px" %)fr-FR 338 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Luxembourg (LU)|(% style="width:213px" %)fr-LU 339 -|(% style="width:172px" %)French (fr)|(% style="width:216px" %)Switzerland (CH)|(% style="width:213px" %)fr-CH 340 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Austria (AT)|(% style="width:213px" %)de-AT 341 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Germany (DE)|(% style="width:213px" %)de-DE 342 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Luxembourg (LU)|(% style="width:213px" %)de-LU 343 -|(% style="width:172px" %)German (de)|(% style="width:216px" %)Switzerland (CH)|(% style="width:213px" %)de-CH 344 -|(% style="width:172px" %)Greek (el)|(% style="width:216px" %)Cyprus (CY)|(% style="width:213px" %)el-CY(*) 345 -|(% style="width:172px" %)Greek (el)|(% style="width:216px" %)Greece (GR)|(% style="width:213px" %)el-GR 346 -|(% style="width:172px" %)Hebrew (iw)|(% style="width:216px" %)Israel (IL)|(% style="width:213px" %)iw-IL 347 -|(% style="width:172px" %)Hindi (hi)|(% style="width:216px" %)India (IN)|(% style="width:213px" %)hi-IN 348 -|(% style="width:172px" %)Hungarian (hu)|(% style="width:216px" %)Hungary (HU)|(% style="width:213px" %)hu-HU 349 -|(% style="width:172px" %)Icelandic (is)|(% style="width:216px" %)Iceland (IS)|(% style="width:213px" %)is-IS 350 -|(% style="width:172px" %)Indonesian (in)|(% style="width:216px" %)Indonesia (ID)|(% style="width:213px" %)in-ID(*) 351 -|(% style="width:172px" %)Irish (ga)|(% style="width:216px" %)Ireland (IE)|(% style="width:213px" %)ga-IE(*) 352 -|(% style="width:172px" %)Italian (it)|(% style="width:216px" %)Italy (IT)|(% style="width:213px" %)it-IT 353 - 354 -~* - [[https:~~/~~/www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale>>https://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html#cldrlocale]] 355 - 356 356 Examples 357 357 358 358 22/06/1981 would be described as dd/MM/YYYY, with locale en-GB 314 + 359 359 2008-mars-12 would be described as YYYY-MMM-DD, with locale fr-FR 316 + 360 360 22 July 1981 would be described as dd MMMM YYYY, with locale en-US 318 + 361 361 22 Jul 1981 would be described as dd MMM YYYY 320 + 362 362 2010 D62 would be described as YYYYDnn (day 62 of the year 2010) 363 363 364 364 The following pattern letters are defined (all other characters from 'A' to 'Z' and from 'a' to 'z' are reserved): 365 365 366 -(% style="width:850.294px" %) 367 -|(% style="width:125px" %)**Letter**|(% style="width:385px" %)**Date or Time Component**|(% style="width:180px" %)**Presentation**|(% style="width:157px" %)**Examples** 368 -|(% style="width:125px" %)G|(% style="width:385px" %)Era designator|(% style="width:180px" %)Text|(% style="width:157px" %)AD 369 -|(% style="width:125px" %)yy|(% style="width:385px" %)Year short (upper case is Year of Week(% contenteditable="false" tabindex="-1" data-macro="startmacro:footnote|-||-|yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years" data-widget="xwiki-macro" class="macro hidden macro-placeholder" %)macro:footnote(% contenteditable="false" tabindex="-1" data-macro="startmacro:footnote|-||-|yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years" data-widget="xwiki-macro" class="macro footnoteRef" id="x_footnote_ref_3" %)^^[[3>>doc:null||anchor="x_footnote_3"]]^^(%%))|(% style="width:180px" %)Year|(% style="width:157px" %)96 370 -|(% style="width:125px" %)yyyy|(% style="width:385px" %)Year Full (upper case is Year of Week)|(% style="width:180px" %)Year|(% style="width:157px" %)1996 371 -|(% style="width:125px" %)MM|(% style="width:385px" %)Month number in year starting with 1|(% style="width:180px" %)Month|(% style="width:157px" %)07 372 -|(% style="width:125px" %)MMM|(% style="width:385px" %)Month name short|(% style="width:180px" %)Month|(% style="width:157px" %)Jul 373 -|(% style="width:125px" %)MMMM|(% style="width:385px" %)Month name full|(% style="width:180px" %)Month|(% style="width:157px" %)July 374 -|(% style="width:125px" %)ww|(% style="width:385px" %)Week in year|(% style="width:180px" %)Number|(% style="width:157px" %)27 375 -|(% style="width:125px" %)W|(% style="width:385px" %)Week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2 376 -|(% style="width:125px" %)DD|(% style="width:385px" %)Day in year|(% style="width:180px" %)Number|(% style="width:157px" %)189 377 -|(% style="width:125px" %)dd|(% style="width:385px" %)Day in month|(% style="width:180px" %)Number|(% style="width:157px" %)10 378 -|(% style="width:125px" %)F|(% style="width:385px" %)Day of week in month|(% style="width:180px" %)Number|(% style="width:157px" %)2 379 -|(% style="width:125px" %)E|(% style="width:385px" %)Day name in week|(% style="width:180px" %)Text|(% style="width:157px" %)Tuesday; Tue 380 -|(% style="width:125px" %)U|(% style="width:385px" %)Day number of week (1 = Monday, ..., 7 = Sunday)|(% style="width:180px" %)Number|(% style="width:157px" %)1 381 -|(% style="width:125px" %)HH|(% style="width:385px" %)Hour in day (0-23)|(% style="width:180px" %)Number|(% style="width:157px" %)0 382 -|(% style="width:125px" %)kk|(% style="width:385px" %)Hour in day (1-24)|(% style="width:180px" %)Number|(% style="width:157px" %)24 383 -|(% style="width:125px" %)KK|(% style="width:385px" %)Hour in am/pm (0-11)|(% style="width:180px" %)Number|(% style="width:157px" %)0 384 -|(% style="width:125px" %)hh|(% style="width:385px" %)Hour in am/pm (1-12)|(% style="width:180px" %)Number|(% style="width:157px" %)12 385 -|(% style="width:125px" %)mm|(% style="width:385px" %)Minute in hour|(% style="width:180px" %)Number|(% style="width:157px" %)30 386 -|(% style="width:125px" %)ss|(% style="width:385px" %)Second in minute|(% style="width:180px" %)Number|(% style="width:157px" %)55 387 -|(% style="width:125px" %)S|(% style="width:385px" %)Millisecond|(% style="width:180px" %)Number|(% style="width:157px" %)978 388 -|(% style="width:125px" %)n|(% style="width:385px" %)((( 325 +|Letter|Date or Time Component|Presentation|Examples 326 +|G|Era designator|[[Text>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]]|AD 327 +|yy|Year short (upper case is Year of Week{{footnote}}yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years{{/footnote}})|[[Year>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#year]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#year]]|96 328 +|yyyy|Year Full (upper case is Year of Week)|Year|1996 329 +|MM|Month number in year starting with 1|Month|07 330 +|MMM|Month name short|Month|Jul 331 +|MMMM|Month name full|Month|July 332 +|ww|Week in year|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|27 333 +|W|Week in month|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|2 334 +|DD|Day in year|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|189 335 +|dd|Day in month|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|10 336 +|F|Day of week in month|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|2 337 +|E|Day name in week|[[Text>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#text]]|Tuesday; Tue 338 +|U|Day number of week (1 = Monday, ..., 7 = Sunday)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|1 339 +|HH|Hour in day (0-23)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|0 340 +|kk|Hour in day (1-24)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|24 341 +|KK|Hour in am/pm (0-11)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|0 342 +|hh|Hour in am/pm (1-12)|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|12 343 +|mm|Minute in hour|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|30 344 +|ss|Second in minute|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|55 345 +|S|Millisecond|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|978 346 +|n|((( 389 389 Number of periods, used after a SDMX 348 + 390 390 Frequency Identifier such as M, Q, D (month, quarter, day) 391 -)))| (% style="width:180px" %)Number|(% style="width:157px" %)12350 +)))|[[Number>>url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]][[url:https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#number]]|12 392 392 393 393 The model is illustrated below: 394 394 395 -(% contenteditable="false" tabindex="-1" %)[[image:1750074822764-573.png||data-widget="image"]] 396 396 355 + 397 397 **Figure 24 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD dimension with the additional information on the component map to describe the time format?** 398 398 399 -(% contenteditable="false" tabindex="-1" %)[[image:1750074865924-797.png||data-widget="image"]] 400 400 401 -(% class="wikigeneratedid" id="HFigure25showinganinputdateformat2CwhoseoutputfrequencyisderivedfromtheoutputvalueoftheFREQDimension" %) 402 -**Figure 25 showing an input date format, whose output frequency is derived from the output value of the FREQ Dimension** 403 403 404 -=== 13.7.2Numericalbaseddatetime ===360 +==== Figure 25 showing an input date format, whose output frequency is derived from the output value of the FREQ Dimension ==== 405 405 406 - Wherethesource datetimeinput is purely numerical,the mapping rules are definedby the **Base** as a valid[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[TimePeriod>>doc:sdmx:Glossary.Time period.WebHome]], and the **Period** which must take one of the following enumerated values:362 +=== 13.3.6 Numerical based datetime === 407 407 364 +Where the source datetime input is purely numerical, the mapping rules are defined by the **Base** as a valid SDMX Time Period, and the **Period** which must take one of the following enumerated values: 365 + 408 408 * day 409 409 * second 410 410 * millisecond ... ... @@ -411,63 +411,75 @@ 411 411 * microsecond 412 412 * nanosecond 413 413 414 -(% style="width:545.294px" %) 415 -|(% style="width:268px" %)Numerical datetime systems|(% style="width:121px" %)Base|(% style="width:153px" %)Period 416 -|(% style="width:268px" %)((( 372 +|Numerical datetime systems|Base|Period 373 +|((( 417 417 Epoch Time (UNIX) 375 + 418 418 Milliseconds since 01 Jan 1970 419 -)))| (% style="width:121px" %)1970|(% style="width:153px" %)millisecond420 -|( % style="width:268px" %)(((377 +)))|1970|millisecond 378 +|((( 421 421 Windows System Time 380 + 422 422 Milliseconds since 01 Jan 1601 423 -)))| (% style="width:121px" %)1601|(% style="width:153px" %)millisecond382 +)))|1601|millisecond 424 424 425 425 The example above illustrates numerical based datetime mapping rules for two commonly used time standards. 426 426 427 427 The model is illustrated below: 428 428 429 - (% contenteditable="false" tabindex="-1" %)[[image:1750074994887-415.png||data-widget="image"]]388 +[[image:1750072341491-790.jpeg]] 430 430 431 -**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use**390 +**Figure 26 showing the component map mapping the SOURCE_DATE Dimension to the** 432 432 433 -=== 13.7.3Mappingmore complextime inputs ===392 +==== TIME_PERIOD Dimension with the additional information on the component map to describe the numerical datetime system in use ==== 434 434 435 - [[VTL>>doc:sdmx:Glossary.Validationand transformationlanguage.WebHome]]should be used formore complex time inputsthat cannot be interpreted using the pattern based on numerical methods.394 +=== Mapping more complex time inputs === 436 436 437 - ==13.8UsingTIME_PERIODinmapping rules==396 +VTL should be used for more complex time inputs that cannot be interpreted using the pattern based on numerical methods. 438 438 439 - ThesourceTIME_PERIOD[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]can be used in conjunction with other input [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] to create discrete mapping ruleswhere the output is conditional on the [[time period>>doc:sdmx:Glossary.Time period.WebHome]] value.398 +== Using TIME_PERIOD in mapping rules == 440 440 441 -The main usecaseissettingthevalueofObservation[[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]]inthetarget[[dataset>>doc:sdmx:Glossary.Dataset.WebHome]].400 +The source TIME_PERIOD Dimension can be used in conjunction with other input Dimensions to create discrete mapping rules where the output is conditional on the time period value. 442 442 443 -(% style="width:709.294px" %) 444 -|(% style="width:103px" %)**Rule**|(% style="width:405px" %)**Source**|(% style="width:198px" %)**Target** 445 -|(% style="width:103px" %)1|(% style="width:405px" %)((( 402 +The main use case is setting the value of Observation Attributes in the target dataset. 403 + 404 +|Rule|Source|Target 405 +|1|((( 446 446 If 407 + 447 447 INDICATOR=XULADS; and TIME_PERIOD=2007. 448 -)))|(% style="width:198px" %)((( 449 -Set OBS_CONF=F 409 +)))|((( 410 +Set 411 + 412 +OBS_CONF=F 450 450 ))) 451 -| (% style="width:103px" %)2|(% style="width:405px" %)(((414 +|2|((( 452 452 If 416 + 453 453 INDICATOR=XULADS; and TIME_PERIOD=2008. 454 -)))| (% style="width:198px" %)Set OBS_CONF=F455 -| (% style="width:103px" %)3|(% style="width:405px" %)(((418 +)))|Set OBS_CONF=F 419 +|3|((( 456 456 If 421 + 457 457 INDICATOR=XULADS; and TIME_PERIOD=2009. 458 -)))|(% style="width:198px" %)((( 459 -Set OBS_CONF=F 423 +)))|((( 424 +Set 425 + 426 +OBS_CONF=F 460 460 ))) 461 -| (% style="width:103px" %)4|(% style="width:405px" %)(((428 +|4|((( 462 462 If 430 + 463 463 INDICATOR=XULADS; and TIME_PERIOD=2010. 464 -)))|(% style="width:198px" %)((( 465 -Set OBS_CONF=**C** 432 +)))|((( 433 +Set 434 + 435 +OBS_CONF=**C** 466 466 ))) 467 467 468 -In the example above, OBS_CONF is an Observation [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]].438 +In the example above, OBS_CONF is an Observation Attribute. 469 469 470 -== 13. 9Time span mapping rules using validity periods ==440 +== 13. Time span mapping rules using validity periods == 471 471 472 472 Creating discrete mapping rules for each TIME_PERIOD is impractical where rules need to cover a specific span of time regardless of frequency, and for high-frequency data. 473 473 ... ... @@ -475,22 +475,25 @@ 475 475 476 476 By specifying validity periods, the example from Section 13.8 can be re-written using two rules as follows: 477 477 478 -(% style="width:516.294px" %) 479 -|(% style="width:74px" %)Rule|(% style="width:215px" %)Source|(% style="width:223px" %)Target 480 -|(% style="width:74px" %)1|(% style="width:215px" %)((( 448 +|Rule|Source|Target 449 +|1|((( 481 481 If 451 + 482 482 INDICATOR=XULADS. 483 - Validity Period484 - start period=2007485 -end period=2009 486 -)))| (% style="width:223px" %)Set OBS_CONF=F487 -| (% style="width:74px" %)2|(% style="width:215px" %)(((453 + 454 + 455 +Validity Period start period=2007 end period=2009 456 +)))|Set OBS_CONF=F 457 +|2|((( 488 488 If 459 + 489 489 INDICATOR=XULADS. 490 -Validity Period 491 -start period=2010 492 -)))|(% style="width:223px" %)((( 461 + 462 + 463 +Validity Period start period=2010 464 +)))|((( 493 493 Set 466 + 494 494 OBS_CONF=F 495 495 ))) 496 496 ... ... @@ -498,132 +498,132 @@ 498 498 499 499 In Rule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time. Any observations reporting data for the Indicator XULADS that fall into that time range will have an OBS_CONF value of C. 500 500 501 - InRule 2, no end period is specified so remains in effect from the start of the period (2010-01-01T00:00:00) until the end of time.Any observations reportingdata for theIndicator XULADS that fall into that time range will havean OBS_CONF value of C.474 +== 13. Mapping examples == 502 502 503 -== 13. 10Mappingexamples==476 +=== 13. Many to one mapping (N-1) === 504 504 505 -=== 13.10.1 Many to one mapping (N3513 -1) === 478 +|Source|Map To 479 +|((( 480 +**FREQ**="A" 506 506 507 -(% style="width:542.294px" %) 508 -|(% style="width:274px" %)**Source**|(% style="width:265px" %)**Map To** 509 -|(% style="width:274px" %)((( 510 -(% style="color:#8e44ad" %)**FREQ**="A" 511 511 ADJUSTMENT="N" 483 + 512 512 **REF_AREA**="PL" 485 + 513 513 **COUNTERPART_AREA**="W0" 487 + 514 514 REF_SECTOR="S1" 489 + 515 515 COUNTERPART_SECTOR="S1" 491 + 516 516 ACCOUNTING_ENTRY="B" 493 + 517 517 STO="B5G" 518 -)))|(% style="width:265px" %)((( 519 -(% style="color:#8e44ad" %)FREQ="A" 495 +)))|((( 496 +FREQ="A" 497 + 520 520 REF_AREA="PL" 499 + 521 521 COUNTERPART_AREA="W0" 522 -INDICATOR="IND_ABC" (%%)501 +INDICATOR="IND_ABC" 523 523 524 524 ))) 525 525 526 -The bold [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]](% style="color:#e74c3c" %)map(%%)from source to target verbatim. The mapping simply specifies:505 +The bold Dimensions map from source to target verbatim. The mapping simply specifies: 527 527 528 -> FREQ => FREQ 529 -> REF_AREA=> REF_AREA 530 -> COUNTERPART_AREA=> COUNTERPART _AREA 507 +FREQ => FREQ 531 531 532 - No [[Representation>>doc:sdmx:Glossary.Representation.WebHome]]Mapping is required. The source value simply copies across unmodified.509 +REF_AREA=> REF_AREA 533 533 534 -T he remaining [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] all (% style="color:#e74c3c" %)map(%%) to the Indicator [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]].This is an example of many [[Dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] mapping to one [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. In this case a [[Representation>>doc:sdmx:Glossary.Representation.WebHome]]Mapping is required, and the mapping first describes the input 'partial key' and how this (% style="color:#e74c3c" %)maps(%%) to the target indicator:511 +COUNTERPART_AREA=> COUNTERPART _AREA 535 535 536 - >N:S1:S1:B:B5G=>IND_ABC513 +No Representation Mapping is required. The source value simply copies across unmodified. 537 537 538 - Wherethekey sequenceisbasedonthe orderspecifiedinthe(%style="color:#e74c3c"%)mapping(%%)(i.e[[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]],REF_SECTOR,etc willresultin the firstvalueNbeingtakenfrom [[ADJUSTMENT>>doc:sdmx:Glossary.Adjustment.WebHome]]asthiswasthe firstiteminthesource[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] list.515 +The remaining Dimensions all map to the Indicator Dimension. This is an example of many Dimensions mapping to one Dimension. In this case a Representation Mapping is required, and the mapping first describes the input 'partial key' and how this maps to the target indicator: 539 539 540 - **Note**:The key order is NOT based on the [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] order of the [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]], as the (% style="color:#e74c3c" %)mapping(%%) needs to be resilient to the [[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]] changing.517 +N:S1:S1:B:B5G => IND_ABC 541 541 542 - ===13.10.2Mappingotherdata types toCodeId===519 +Where the key sequence is based on the order specified in the mapping (i.e ADJUSTMENT, REF_SECTOR, etc will result in the first value N being taken from ADJUSTMENT as this was the first item in the source Dimension list. 543 543 544 - Inthecase wheretheincomingdata type isnotatring and not a [[code>>doc:sdmx:Glossary.Code.WebHome]]identifieri.e.thesource [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]isof type Integer and thetargetisCodelist. This is supportedby the RepresentationMap. The RepresentationMapsource canreference a Codelist,Valuelist,orbefree text,thefreetextcaninclude regular expressions.521 +**Note**: The key order is NOT based on the Dimension order of the DSD, as the mapping needs to be resilient to the DSD changing. 545 545 546 -The following [[representation>>doc:sdmx:Glossary.Representation.WebHome]] (% style="color:#e74c3c" %)mapping(%%) can be used to explicitly (% style="color:#e74c3c" %)map(%%) each [[age>>doc:sdmx:Glossary.Age.WebHome]] to an output [[code>>doc:sdmx:Glossary.Code.WebHome]]. 523 +1. 524 +11. 525 +111. Mapping other data types to Code Id 547 547 548 -(% style="width:402.294px" %) 549 -|(% style="width:197px" %)**Source Input Free Text**|(% style="width:204px" %)**Desired Output Code Id** 550 -|(% style="width:197px" %)0|(% style="width:204px" %)A 551 -|(% style="width:197px" %)1|(% style="width:204px" %)A 552 -|(% style="width:197px" %)2|(% style="width:204px" %)A 553 -|(% style="width:197px" %)3|(% style="width:204px" %)B 554 -|(% style="width:197px" %)4|(% style="width:204px" %)B 527 +In the case where the incoming data type is not a string and not a code identifier i.e. the source Dimension is of type Integer and the target is Codelist. This is supported by the RepresentationMap. The RepresentationMap source can reference a Codelist, Valuelist, or be free text, the free text can include regular expressions. 555 555 529 +The following representation mapping can be used to explicitly map each age to an output code. 530 + 531 +|Source Input Free Text|Desired Output Code Id 532 +|0|A 533 +|1|A 534 +|2|A 535 +|3|B 536 +|4|B 537 + 556 556 If this mapping takes advantage of regular expressions it can be expressed in two rules: 557 557 558 -(% style="width:336.294px" %) 559 -|(% style="width:182px" %)**Regular Expression**|(% style="width:151px" %)**Desired Output** 560 -|(% style="width:182px" %)[0-2]|(% style="width:151px" %)A 561 -|(% style="width:182px" %)[3-4]|(% style="width:151px" %)B 562 562 563 - === 13.10.3 ObservationAttributesforTimePeriod===541 +Regular Expression Desired Output 564 564 565 -This use case is where a specific observation for a specific [[time period>>doc:sdmx:Glossary.Time period.WebHome]] has an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] value. 543 +|[0-2]|A 544 +|[3-4]|B 566 566 567 -(% style="width:621.294px" %) 568 -|(% style="width:201px" %)Input INDICATOR|(% style="width:192px" %)Input TIME_PERIOD|(% style="width:225px" %)Output OBS_CONF 569 -|(% style="width:201px" %)XULADS|(% style="width:192px" %)2008|(% style="width:225px" %)C 570 -|(% style="width:201px" %)XULADS|(% style="width:192px" %)2009|(% style="width:225px" %)C 571 -|(% style="width:201px" %)XULADS|(% style="width:192px" %)2010|(% style="width:225px" %)C 546 +=== 13. Observation Attributes for Time Period === 572 572 573 - Orusingavalidityperiod on the[[Representation>>doc:sdmx:Glossary.Representation.WebHome]]Mapping:548 +This use case is where a specific observation for a specific time period has an attribute value. 574 574 575 -(% style="width:629.294px" %) 576 -|(% style="width:202px" %)Input INDICATOR|(% style="width:197px" %)Valid From/ Valid To|(% style="width:227px" %) Output OBS_CONF 577 -|(% style="width:202px" %)XULADS|(% style="width:197px" %) 2008/2010|(% style="width:227px" %)С 550 +|Input INDICATOR|Input TIME_PERIOD|Output OBS_CONF 551 +|XULADS|2008|C 552 +|XULADS|2009|C 553 +|XULADS|2010|C 578 578 579 - ===13.10.4Timemapping===555 +Or using a validity period on the Representation Mapping: 580 580 581 -T hisusecaseistocreatea[[timeperiod>>doc:sdmx:Glossary.Time period.WebHome]]fromaninputthatdoesnotrespect[[SDMX>>doc:sdmx:Glossary.Statisticaldataandmetadataexchange.WebHome]]TimeFormats.557 +Input INDICATOR Valid From/ Valid To Output OBS_CONF 582 582 583 - The[[Component>>doc:sdmx:Glossary.Component.WebHome]]MappingfromSYS_TIMEtoTIME_PERIODspecifiesitselfasatimemappingwiththefollowingdetails:559 +XULADS 2008/2010 C 584 584 585 -(% style="width:652.294px" %) 586 -|(% style="width:139px" %)Source Value|(% style="width:165px" %)Source Mapping|(% style="width:182px" %)Target Frequency|(% style="width:163px" %)Output 587 -|(% style="width:139px" %)18/07/1981|(% style="width:165px" %)dd/MM/yyyy|(% style="width:182px" %)A|(% style="width:163px" %)1981 561 +=== 13. Time mapping === 588 588 589 - Whenthetargetfrequencyisbasedonanothertarget[[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]value,inhisexamplethe value oftheFREQ [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]]inhetarget[[DSD>>doc:sdmx:Glossary.Datastructure definition.WebHome]].563 +This use case is to create a time period from an input that does not respect SDMX Time Formats. 590 590 591 -(% style="width:658.294px" %) 592 -|(% style="width:143px" %)Source Value|(% style="width:163px" %) Source Mapping|(% style="width:176px" %)Target Dimension|(% style="width:173px" %)Frequency Output 593 -|(% style="width:143px" %)18/07/1981|(% style="width:163px" %)dd/MM/yyyy|(% style="width:176px" %)FREQ|(% style="width:173px" %)1981-07-18 (when FREQ=D) 565 +The Component Mapping from SYS_TIME to TIME_PERIOD specifies itself as a time mapping with the following details: 594 594 595 - When the source is a numerical format. 567 +|Source Value|Source Mapping|Target Frequency|Output 568 +|18/07/1981|dd/MM/yyyy|A|1981 596 596 597 -(% style="width:658.294px" %) 598 -|(% style="width:143px" %)Source Value|(% style="width:163px" %) Start Period|(% style="width:176px" %)Interval|(% style="width:176px" %)Target FREQ|(% style="width:173px" %) Output 599 -|(% style="width:143px" %)1589808220|(% style="width:163px" %)1970|(% style="width:176px" %) millisecond|(% style="width:176px" %)M|(% style="width:173px" %)2020-05 570 +When the target frequency is based on another target Dimension value, in this example the value of the FREQ Dimension in the target DSD. 600 600 572 +Source Value Source Mapping Target Frequency Output 573 + 574 +Dimension 575 + 576 +|18/07/1981 dd/MM/yyyy FREQ| |1981-07-18 (when FREQ=D) 577 +| When the source is a numerical format| | 578 +|Source Value Start Period Interval|((( 579 +Target 580 + 581 +FREQ 582 +)))|Output 583 +|1589808220 1970 millisecond|M|2020-05 584 + 601 601 When the source frequency is lower than the target frequency additional information 3568 can be provided for resolve to start of period, end of period, or mid period, as shown 3569 in the following example: 602 602 603 -(% style="width:666.294px" %) 604 -|(% style="width:131px" %) Source Value|(% style="width:143px" %)Source Mapping|(% style="width:149px" %)Target Dimension|(% style="width:114px" %)Frequency|(% style="width:126px" %)Output 605 -|(% style="width:131px" %)1981|(% style="width:143px" %)yyyy|(% style="width:149px" %)D – End of Period|(% style="width:114px" %) |(% style="width:126px" %)1981-12-31 587 + Source Value Source Mapping Target Frequency Output 606 606 607 - Whenthestart of yearis April 1^^st^^ the Structure (% style="color:#e74c3c" %)Map(%%) has YearStart=04-01:589 +Dimension 608 608 609 -(% style="width:666.294px" %) 610 -|(% style="width:131px" %) Source Value|(% style="width:143px" %)Source Mapping|(% style="width:149px" %)Target Dimension|(% style="width:114px" %)Frequency|(% style="width:126px" %)Output 611 -|(% style="width:131px" %)1981|(% style="width:143px" %)yyyy|(% style="width:149px" %)D – End of Period|(% style="width:114px" %) |(% style="width:126px" %)1982-03-31 591 + 1981 yyyy D – End of Period 1981-12-31 612 612 613 ----- 614 614 615 -(% contenteditable="false" tabindex="-1" %) 616 -((( 617 -(% class="macro" data-macro="startmacro:putFootnotes|-|" data-widget="xwiki-macro" %) 618 -((( 619 -(% class="macro-placeholder hidden" %) 620 -((( 621 -macro:putFootnotes 622 -))) 594 +When the start of year is April 1^^st^^ the Structure Map has YearStart=04-01: 623 623 624 -(% class="footnotes" %) 625 -1. [[^>>doc:null||anchor="x_footnote_ref_1" id="x_footnote_1" class="footnoteBackRef"]] Unidimensional datasets are those with a single 'indicator' or 'series code' dimension. 626 -1. [[^>>doc:null||anchor="x_footnote_ref_2" id="x_footnote_2" class="footnoteBackRef"]] A list of commonly used locales can be found in the Java supported locales: https~://www.oracle.com/java/technologies/javase/jdk8-jre8-suported-locales.html// 627 -1. [[^>>doc:null||anchor="x_footnote_ref_3" id="x_footnote_3" class="footnoteBackRef"]] yyyy represents the calendar year while YYYY represents the year of the week, which is only relevant for 53 week years 628 -))) 629 -))) 596 + Source Value Source Mapping Target Frequency Output 597 + 598 +Dimension 599 + 600 +---- 601 + 602 +{{putFootnotes/}}
- 1750074822764-573.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -25.1 KB - Content
- 1750074865924-797.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -46.8 KB - Content
- 1750074994887-415.png
-
- Author
-
... ... @@ -1,1 +1,0 @@ 1 -xwiki:XWiki.helena - Size
-
... ... @@ -1,1 +1,0 @@ 1 -12.4 KB - Content