Skip to Content
Last modified by Artur K. on 2026/05/29 14:28
From version 1.9
edited by Helena K.
on 2026/01/15 12:39
Change comment: There is no comment for this version
To version 3.3
edited by Helena K.
on 2026/01/15 13:11
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -61,7 +61,7 @@
61 61  
62 62  === 2.1.3 Suitability of available DSDs and code lists ===
63 63  
64 -In case an existing DSD is close to but differs from what is needed, it may: (i) contain irrelevant concepts, (ii) lack some required concepts, (iii) use the concepts in different roles than required, (iv) deviate with respect to some of the code lists, or (v) contain pure dimensions when mixed dimensions would make more sense or vice versa. More complex situations that are combinations of several (or even all) of these five cases may occur as well. For example, an existing DSD could contain unnecessary concepts and lack other concepts at the same time.
64 +In case an existing DSD is close to but differs from what is needed, it may: {{{(i)}}} contain irrelevant concepts, (ii) lack some required concepts, (iii) use the concepts in different roles than required, (iv) deviate with respect to some of the code lists, or (v) contain pure dimensions when mixed dimensions would make more sense or vice versa. More complex situations that are combinations of several (or even all) of these five cases may occur as well. For example, an existing DSD could contain unnecessary concepts and lack other concepts at the same time.
65 65  
66 66  ==== 2.1.3.1 Irrelevant concepts ====
67 67  
... ... @@ -243,31 +243,27 @@
243 243  
244 244  The decision on content and number of concepts in a DSD usually leads to the question of how far the “//indicator//” dimension should be decomposed. There are some (cross-domain) concepts, such as geographical and temporal reference and unit of measure, that are relevant in most DSDs. Once those are defined (the usage of the SDMX COG is highly recommended!) the actual “//subject-matter//” or “//domain//” concepts remain. One option is to combine all those concepts into one “indicator” dimension which may make sense in certain scenarios, for example for smaller single-domain, single-purpose DSDs with few or no crossclassifications or for display in an end-user dissemination tool. The other extreme strategy is to decompose into as many components as possible by splitting any breakdown concepts from the core indicator concept.
245 245  
246 -The range of options between the “//just one//”// //(mixed) and “//all component//” subject-matter dimensions approaches is subject to the comprehensiveness (i.e. size, coverage) of the data exchange that the DSD is being developed for. If using a “mixed dimensions” approach, rules for the composition of the mixed dimension(s) may be specified (e.g. concatenate concepts A, B, and C to get mixed dimension X), allowing their easy re-decomposition. In general composite dimensions should be avoided as previously recommended by the SDMX Technical Notes, but there are cases that suggest the usage of composite dimensions. Table 4 juxtaposes general pros and cons of the “//many pure concepts//” and “//fewer composite concepts//” approaches.
246 +The range of options between the “//just one//” (mixed) and “//all component//” subject-matter dimensions approaches is subject to the comprehensiveness (i.e. size, coverage) of the data exchange that the DSD is being developed for. If using a “mixed dimensions” approach, rules for the composition of the mixed dimension(s) may be specified (e.g. concatenate concepts A, B, and C to get mixed dimension X), allowing their easy re-decomposition. In general composite dimensions should be avoided as previously recommended by the SDMX Technical Notes, but there are cases that suggest the usage of composite dimensions. Table 4 juxtaposes general pros and cons of the “//many pure concepts//” and “//fewer composite concepts//” approaches.
247 247  
248 248  **Table 4. General comparison of data structuring approaches**
249 249  
250 -|(% style="width:360px" %)**Many pure concepts**|(% style="width:1255px" %)**Few composite concepts**
251 -|(% style="width:360px" %)cleaner data structure|(% style="width:1255px" %)(((
250 +|(% style="width:416px" %)**Many pure concepts**|(% style="width:1199px" %)**Few composite concepts**
251 +|(% style="width:416px" %)cleaner data structure|(% style="width:1199px" %)(((
252 252  Mixed dimensions may be composed inconsistently making the decomposition into purer concepts and code lists difficult (requiring complex mapping etc.). Information that corresponds to the same concept may be included in different dimensions, e.g. reference year is contained in the indicator dimension in the first example but in the unit in the second example below. The optimal common data structure would consist of Economic Indicator, Unit, and Base period.
253 253  
254 254  [[image:1768469652632-803.png||height="106" width="352"]]
255 -
256 -
257 257  )))
258 -|(% style="width:360px" %)shorter and simpler code lists|(% style="width:1255px" %)code lists longer and more complex, may require hierarchy to be “readable”
259 -|(% style="width:360px" %)more flexible in terms of defining constraints, but constraints more complex|(% style="width:1255px" %)simpler constraints, but some constraints may be difficult to be represented because of mixed dimensions. Consider for instance a constraint “Base period = 1995” in the above example, where some observations include the base period in the Economic Indicator dimension, others in the Unit dimension. Instead of specifying a constraint on a pure Base Period dimension, the constraints may have to be specified at observation (or time series) level
260 -|(% style="width:360px" %)more flexible in terms of mapping to other data structures (used by other systems), further processing and analysis (e.g. tabulation, dissemination format), and future needs|(% style="width:1255px" %)“mixed” dimensions make data structure less flexible in these respects
261 -|(% style="width:360px" %)longer (i.e. more complex) observation keys|(% style="width:1255px" %)shorter keys
262 -|(% style="width:360px" %)special values of code lists such as “not applicable”, “total” may be rather heavily used|(% style="width:1255px" %)less usage of these special values
263 -|(% style="width:360px" %)creates sparse data if many observations use “not applicable”|(% style="width:1255px" %)way to avoid sparseness
264 -|(% style="width:360px" %)many constraints may be necessary due to sparseness|(% style="width:1255px" %)typically fewer constraints required because data are less sparse
265 -|(% style="width:360px" %)many dimensions are tantamount to many attachment levels for attributes (i.e. DSD more flexible in terms of attribute attachment)|(% style="width:1255px" %)less dimensions = less possible attribute attachment levels
266 -|(% style="width:360px" %)more difficult to handle by an end user|(% style="width:1255px" %)presumably more easily comprehensible and manageable by an end user
267 -|(% style="width:360px" %)more flexible in terms of defining queries; can be mapped to any “mixed” representation|(% style="width:1255px" %)less flexible in terms of search and retrieval
256 +|(% style="width:416px" %)shorter and simpler code lists|(% style="width:1199px" %)code lists longer and more complex, may require hierarchy to be “readable”
257 +|(% style="width:416px" %)more flexible in terms of defining constraints, but constraints more complex|(% style="width:1199px" %)simpler constraints, but some constraints may be difficult to be represented because of mixed dimensions. Consider for instance a constraint “Base period = 1995” in the above example, where some observations include the base period in the Economic Indicator dimension, others in the Unit dimension. Instead of specifying a constraint on a pure Base Period dimension, the constraints may have to be specified at observation (or time series) level
258 +|(% style="width:416px" %)more flexible in terms of mapping to other data structures (used by other systems), further processing and analysis (e.g. tabulation, dissemination format), and future needs|(% style="width:1199px" %)“mixed” dimensions make data structure less flexible in these respects
259 +|(% style="width:416px" %)longer (i.e. more complex) observation keys|(% style="width:1199px" %)shorter keys
260 +|(% style="width:416px" %)special values of code lists such as “not applicable”, “total” may be rather heavily used|(% style="width:1199px" %)less usage of these special values
261 +|(% style="width:416px" %)creates sparse data if many observations use “not applicable”|(% style="width:1199px" %)way to avoid sparseness
262 +|(% style="width:416px" %)many constraints may be necessary due to sparseness|(% style="width:1199px" %)typically fewer constraints required because data are less sparse
263 +|(% style="width:416px" %)many dimensions are tantamount to many attachment levels for attributes (i.e. DSD more flexible in terms of attribute attachment)|(% style="width:1199px" %)less dimensions = less possible attribute attachment levels
264 +|(% style="width:416px" %)more difficult to handle by an end user|(% style="width:1199px" %)presumably more easily comprehensible and manageable by an end user
265 +|(% style="width:416px" %)more flexible in terms of defining queries; can be mapped to any “mixed” representation|(% style="width:1199px" %)less flexible in terms of search and retrieval
268 268  
269 -
270 -
271 271  The latter two aspects mentioned in the table could be summarized as the “many pure dimensions” approach being more difficult to handle for a “basic” user, but providing fewer options for an “advanced” user. When it comes to dissemination to end users, a purer data structure is the appropriate format for consumption by applications and advanced users. For less advanced user groups it makes sense to hide the (for them: unnecessary) complexity by means of concatenating dimensions, for instance to create a time series view.
272 272  
273 273  Comparing single-purpose and single-domain exchange scenarios with multi-domain and/or multi-purpose scenarios, pure concepts are typically easier to achieve in the former, whereas composite concepts/dimensions may make life easier in the latter, especially because certain cross-classification concepts may only apply to some domains and/or purposes covered. “Purpose” mean“mixed” dimensions make data structure less
... ... @@ -280,18 +280,13 @@
280 280  |**Level of data exchange**|**Pure vs. composite concepts approach**
281 281  |**within an organization**|(((
282 282  Depends on diversity of systems involved in data exchange.
283 -
284 284  The approach that requires the least mapping (and similar processing) steps between the two communicating data structures is preferable in terms of a “quick win” solution.
285 -
286 286  In general, a more granular model is preferable due to its flexibility that helps support potential future needs (with respect to processing, analysis, exchange, dissemination, etc.).
287 -
288 288  However, an internal exchange should not be made more complex than necessary. If the structures of the communicating systems are comparable, it may not make sense to create an artificial intermediary structure that is more pure, but also more complex than both underlying structures.
289 -
290 290  Still, as a longer-term strategy it seems reasonable to define a set of internal “standard” code lists that all systems can map to. This allows bilateral communication via the shared concepts and code lists meaning that every data structure only has to be mapped once – to the internal standard – to be able to communicate with all other participating (i.e. mapped) systems.
291 291  )))
292 292  |**between organizations at national level**|(((
293 293  The pros and cons at this level of exchange are comparable to those at the “within organization” level. If the data structures of the communicating systems are comparable, there is no need to introduce complexity by a conceptually optimal, pure data structure. However, if the data structures deviate to a greater extent (and they often do), they should both be decomposed to find a “common denominator”, a more granular “exchange vocabulary” which they can be mapped to.
294 -
295 295  If related international or national standards exist, they should be used, even though national labels and/or additional levels of detail may be required in the code lists.
296 296  )))
297 297  |**between international organization and national organizations of member countries**|International organizations should collect data at a level of granularity and purity that is most suitable for the intended (and potential future) analyses. The tradeoff with the higher complexity of constraints required to check structural validity of collected data needs to be taken into account as well. Also it is recommended to consider the burden that a more complex data structure may put on national data providers. However, once a DSD is defined, its lifetime is expected to be a number of years. The main effort of the data provider is to specify the mapping from the production data structure to the DSD. Once this is done the data exchange can be automated and the complexity of the DSD does not matter that much.
... ... @@ -305,15 +305,14 @@
305 305  
306 306  **Table 6. Data structuring approaches by role in data exchange**
307 307  
308 -|**Role in data exchange**|**Pure vs. composite concepts approach**
309 -|**Data provider**|(((
299 +|(% style="width:215px" %)**Role in data exchange**|(% style="width:1400px" %)**Pure vs. composite concepts approach**
300 +|(% style="width:215px" %)**Data provider**|(% style="width:1400px" %)(((
310 310  If the composition of the concepts in the data provider's production system largely differs from the one in the DSD, mapping it to a few composite concepts may be more complex than mapping it to many pure concepts. (Mapping to just one mixed concept is straightforward, though.) This is due to the need to decompose and recombine concepts in case of a “mixed concepts” DSD. If the data provider’s internal data structure is very granular or very similar to the DSD, it does not make a huge difference if the concepts in that DSD are pure or not.
311 -
312 312  For a “final” data provider disseminating data to the public, the flexibility offered by a pure data structure in terms of defining different output formats may be beneficial.
313 313  )))
314 -|**Data collector**|Defining constraints for data validation is more complex for a highdimensional, pure DSD. But such a DSD provides more flexibility in terms of consumption and reuse, i.e. mapping to the data collector’s internal data model mapping easier.
315 -|**DSD maintenance**|Pure concepts usually have shorter, less complex code lists and are thus easier to maintain. In contrast, the maintenance of constraints, hierarchical code lists, and derived, composite concepts (e.g. for dissemination) requires more effort.
316 -|**End user (“the public”)**|Consumption and reuse are more flexible in a pure data structure, but it is more difficult to identify observation keys that actually have data because of the created sparseness. (Constraints may help in this respect.) Frequent occurrences of “non applicable” values may also make data usage cumbersome.
304 +|(% style="width:215px" %)**Data collector**|(% style="width:1400px" %)Defining constraints for data validation is more complex for a highdimensional, pure DSD. But such a DSD provides more flexibility in terms of consumption and reuse, i.e. mapping to the data collector’s internal data model mapping easier.
305 +|(% style="width:215px" %)**DSD maintenance**|(% style="width:1400px" %)Pure concepts usually have shorter, less complex code lists and are thus easier to maintain. In contrast, the maintenance of constraints, hierarchical code lists, and derived, composite concepts (e.g. for dissemination) requires more effort.
306 +|(% style="width:215px" %)**End user (“the public”)**|(% style="width:1400px" %)Consumption and reuse are more flexible in a pure data structure, but it is more difficult to identify observation keys that actually have data because of the created sparseness. (Constraints may help in this respect.) Frequent occurrences of “non applicable” values may also make data usage cumbersome.
317 317  
318 318  == 4.2 Number and relations of DSDs ==
319 319  
... ... @@ -325,7 +325,7 @@
325 325  
326 326  The “one DSD” approach works best for single-domain and/or single-purpose scenarios. In more complex scenarios, more complex approaches are more suitable. Usage of the “one DSD” approach in a multi-domain or multi-purpose scenario actually means that one master DSD containing all concepts, code lists, and codes relevant in any (but most likely not all) domains and/or purposes is used by all domains and/or purposes without constraints. If a “many pure concepts” approach is used, the DSD will be sparse and require many “not applicable” values or structure maps.
327 327  
328 -In those more complex scenarios, multi-DSD approaches have more potential. The “master DSD + satellite DSDs” approach imposes more restrictions and aims at a higher degree of content harmonization than the more loosely coupled (or even independent) multi-DSD approach. While the former specifies the concepts and code lists to be used by all derived DSDs, the latter is more flexible. Therefore, the master + satellites approach is suggested for data exchange scenarios with a high degree of harmonization / standardization required such as at the international level or between national and international organizations. Please note that what is termed “master DSD + satellite DSDs” approach here may also be implemented as master DSD plus constrained data flows with or without using structure maps.
318 +In those more complex scenarios, multi-DSD approaches have more potential. The “master DSD + satellite DSDs” approach imposes more restrictions and aims at a higher degree of content harmonization than the more loosely coupled (or even independent) multi-DSD approach. While the former specifies the concepts and code lists to be used by all derived DSDs, the latter is more flexible. Therefore, the master + satellites approach is suggested for data exchange scenarios with a high degree of harmonization/standardization required such as at the international level or between national and international organizations. Please note that what is termed “master DSD + satellite DSDs” approach here may also be implemented as master DSD plus constrained data flows with or without using structure maps.
329 329  
330 330  Even in the multiple independent DSDs approach, sharing of concepts and code lists by reference is recommended. This may be problematic if additional codes are needed by certain DSDs, as neither the addition of codes to a code list used by reference nor the concatenation of multiple code lists included by reference is supported by the current SDMX Technical Standards. The only way of implementing “combined” code lists by reference is to reference each single code from each relevant partial code list.
331 331  
... ... @@ -335,36 +335,22 @@
335 335  
336 336  **Table 7. Data structuring approaches by level of data exchange**
337 337  
338 -|**Level of data exchange**|**Data structuring approa one DSD**|(% colspan="2" %)(((
339 -**ch**
340 -
341 -**master + satellite DSDs**
342 -)))|**multiple, indep. DSDs**
328 +|(% colspan="1" rowspan="2" %)**Level of data exchange**|(% colspan="4" rowspan="1" %)**Data structuring approach**
329 +|**one DSD**|(% colspan="2" %)**master + satellite DSDs**|**multiple, indep. DSDs**
343 343  |**within organization**|(((
344 -best for single-domain, single-purpose can be created on the
345 -
346 -fly from structured databases
331 +best for single-domain, single-purpose can be created on the fly from structured databases
347 347  )))|(% colspan="2" %)use if harmonization is important in covered domains or purposes or if such a set of DSDs is already available at international level|easier to do than master + satellite approach each domain/purpose can maintain DSDs independently can be created on the fly from structured databases
348 348  |**between national organizations**|(% colspan="4" %)the same applies as to the “within organization” scenario
349 -|**Level of data exchange**|(% colspan="3" %)(((
350 -**Data structuring approach**
351 -
352 -**one DSD master + satellite DSDs**
353 -)))|**multiple, indep. DSDs**
354 354  |**between int. organization and national organizations**|(% colspan="2" %)best for single domain, single purpose scenarios that are usually rather restricted with very clear specification of what needs to be exchanged|preferable over multiDSD approach in case of multi-domain and/or multi-purpose scenarios with highly correlated data flows for maintenance reasons|(((
355 -for multi-domain and/or multipurpose scenarios; only recommended if overlap of domains/purposes is minor (e.g. just w.r.t. cross-domain concepts)
356 -
357 -equivalent to multiple “one DSD” solutions, one for each domain / purpose
335 +for multi-domain and/or multipurpose scenarios; only recommended if overlap of domains/purposes is minor (e.g. just w.r.t. cross-domain concepts) equivalent to multiple “one DSD” solutions, one for each domain/purpose
358 358  )))
359 359  |**between international organizations**|(% colspan="3" %)comparable to “national to international” scenario|
360 360  |**dissemination to public**|(% colspan="2" %)for single-domain, single-purpose cases in more complex cases this may be the preferable approach for data discovery tools (one data structure to find and access all data)|(% colspan="2" %)(((
361 361  in multi-purpose or –domain scenarios:
362 362  
363 -if it is relevant for the public to see the relationship between the data structures: use master + satellites approach
364 -
365 -otherwise the multi-DSD option is preferable, although with the highest possible degree of re-use of code lists and concepts
366 -
367 -in both cases: important to include only concepts, code lists, and codes actually available / used by the data
341 +* if it is relevant for the public to see the relationship between the data structures: use master + satellites approach
342 +* otherwise the multi-DSD option is preferable, although with the highest possible degree of re-use of code lists and concepts
343 +* in both cases: important to include only concepts, code lists, and codes actually available/used by the data
368 368  )))
369 369  
370 370  In general, finding the “perfect” data structure is less important for bilateral data exchange. Independent, custom-tailored DSDs may do the job quite well, as harmonization and standardization are typically not of high importance. If the data exchange is just a part of a more comprehensive scenario (e.g. multi-purpose, multi-domain, gateway, or data-sharing scenarios), a master DSD with satellite DSDs is preferable.
... ... @@ -373,20 +373,17 @@
373 373  
374 374  **Table 8. Data structuring approaches by role in data exchange**
375 375  
376 -|**Role in data exchange**|**One DSD vs. master + satellite DSDs vs. multiple, indep. DSDs**
377 -|**Data provider**|It is easier to set up a data submission process against a single DSD (= less initial costs) than against multiple DSDs.
378 -|**Data collector**|(((
352 +|(% style="width:216px" %)**Role in data exchange**|(% style="width:1399px" %)**One DSD vs. master + satellite DSDs vs. multiple, indep. DSDs**
353 +|(% style="width:216px" %)**Data provider**|(% style="width:1399px" %)It is easier to set up a data submission process against a single DSD (= less initial costs) than against multiple DSDs.
354 +|(% style="width:216px" %)**Data collector**|(% style="width:1399px" %)(((
379 379  Data validation is easier with DSDs that only cover what needs to be collected. This is achieved via constraints in the master + satellites approach or via tailor-made independent DSDs. If a single DSD is used in a multi-domain or –purpose scenario, necessary constraints can be specified in the data flow definition or data provision agreement.
380 -
381 381  Further processing of collected data is more flexible and easier if relations are transparent and code lists are shared as in the one DSD or master + satellite DSDs approaches. The “shared context” created through the master DSD increases harmonization and standardization and this way facilitates combined usage of data.
382 382  )))
383 -|**Role in data exchange**|**One DSD vs. master + satellite DSDs vs. multiple, indep. DSDs**
384 -|**DSD maintenance**|(((
358 +|(% style="width:216px" %)**DSD maintenance**|(% style="width:1399px" %)(((
385 385  The complexity and initial costs for developing and maintaining master + satellite DSDs are higher than for independent DSDs as this involves managing constraints and managing impacts of changes in shared code lists to all DSDs.
386 -
387 387  In the multiple independent DSDs approach, development and maintenance efforts may be distributed. This can be seen as an advantage, but on the other hand requires coordination in case the DSDs are only partially independent (i.e. share some code lists).
388 388  )))
389 -|**End user (“the public”)**|For data discovery and retrieval the user needs to know what data is actually available (instead of what might be collected/disseminated with a certain data structure). This means that the potential sparseness should be hidden from the user. A reduced DSD derived from the data structure used in the background is more useful in most cases. Whether this is done via one DSD and constraints, master + satellite DSDs, or independent DSDs does not matter that much for the user.
362 +|(% style="width:216px" %)**End user (“the public”)**|(% style="width:1399px" %)For data discovery and retrieval the user needs to know what data is actually available (instead of what might be collected/disseminated with a certain data structure). This means that the potential sparseness should be hidden from the user. A reduced DSD derived from the data structure used in the background is more useful in most cases. Whether this is done via one DSD and constraints, master + satellite DSDs, or independent DSDs does not matter that much for the user.
390 390  
391 391  = 5 MINIMUM STRUCTURAL AND SEMANTIC REQUIREMENTS =
392 392  
... ... @@ -394,7 +394,7 @@
394 394  
395 395  Certain concepts can be broadly agreed upon as being relevant in any data exchange, although their roles may differ between scenarios. The SDMX Content-Oriented Guidelines define many of these cross-domain concepts and, thus, should be referred to for further details on their specification.
396 396  
397 -In general, multi-purpose and multi-domain scenarios may require more concepts than single-purpose and/or –domain scenarios. This mainly applies to subject-matter (or domainspecific) concepts and concepts that inform about the data source, provider, or process.
370 +In general, multi-purpose and multi-domain scenarios may require more concepts than single-purpose and/or – domain scenarios. This mainly applies to subject-matter (or domainspecific) concepts and concepts that inform about the data source, provider, or process.
398 398  
399 399  Exchanges between organizations, especially on an international level, typically require more concepts to cover context information, as data are transferred out of their usual context, meaning that users in the new context do not have the same knowledge of the data and may need additional background information. For exchanges of data within an organization, some context information may be common (implicit) knowledge so that it does not need to be made explicit in the data structure.
400 400  
... ... @@ -416,19 +416,20 @@
416 416  
417 417  **Table 9. Minimum requirements for DSDs~*~***
418 418  
419 -|**Question**|**Concept**|**COG**|**Code list**|**Time series Cross-section**
420 -|Where?|reference area|X|revision|mand. attribute or dimension
421 -|What?|“indicator”|-|domain|one or multiple dimensions
422 -|How?|unit of measure|X|development|mand. attribute or dimension
423 -|How?|unit multiplier|X|available|mandatory attribute
424 -|How?|decimals|X|available|mandatory attribute
425 -|How?|//adjustment//|X|development|mand. att. not relevant
426 -|When?|time period|X|format|dimension mand. att.
427 -|When?|time format|X|available|mandatory attribute
428 -|When?|time period – collection|X|development|mand. att. cond. att.
429 -|When?|data update – last update|X|time stamp|mandatory attribute
430 -|How often?|//frequency//|X|available|mand. att. or not relevant
431 -|(% colspan="2" %)How much? observation value|-|numeric|dimension measure
392 +(% style="width:1308.83px" %)
393 +|(% style="width:205px" %)**Question**|(% style="width:272px" %)**Concept**|(% style="width:178px" %)**COG**|(% style="width:270px" %)**Code list**|(% style="width:290px" %)**Time series**|(% style="width:221px" %)**Cross-section**
394 +|(% style="width:205px" %)Where?|(% style="width:272px" %)reference area|(% style="width:178px" %)X|(% style="width:270px" %)revision|(% colspan="2" rowspan="1" style="width:478px" %)mand. attribute or dimension
395 +|(% style="width:205px" %)What?|(% style="width:272px" %)“indicator”|(% style="width:178px" %)-|(% style="width:270px" %)domain|(% colspan="2" rowspan="1" style="width:478px" %)one or multiple dimensions
396 +|(% style="width:205px" %)How?|(% style="width:272px" %)unit of measure|(% style="width:178px" %)X|(% style="width:270px" %)development|(% colspan="2" rowspan="1" style="width:478px" %)mand. attribute or dimension
397 +|(% style="width:205px" %)How?|(% style="width:272px" %)unit multiplier|(% style="width:178px" %)X|(% style="width:270px" %)available|(% colspan="2" rowspan="1" style="width:478px" %)mandatory attribute
398 +|(% style="width:205px" %)How?|(% style="width:272px" %)decimals|(% style="width:178px" %)X|(% style="width:270px" %)available|(% colspan="2" rowspan="1" style="width:478px" %)mandatory attribute
399 +|(% style="width:205px" %)How?|(% style="width:272px" %)//adjustment//|(% style="width:178px" %)X|(% style="width:270px" %)development|(% style="width:290px" %)mand. att.|(% style="width:221px" %) not relevant
400 +|(% style="width:205px" %)When?|(% style="width:272px" %)time period|(% style="width:178px" %)X|(% style="width:270px" %)format|(% style="width:290px" %)dimension|(% style="width:221px" %)mand. att.
401 +|(% style="width:205px" %)When?|(% style="width:272px" %)time format|(% style="width:178px" %)X|(% style="width:270px" %)available|(% colspan="2" rowspan="1" style="width:478px" %)mandatory attribute
402 +|(% style="width:205px" %)When?|(% style="width:272px" %)time period – collection|(% style="width:178px" %)X|(% style="width:270px" %)development|(% style="width:290px" %)mand. att.|(% style="width:221px" %)cond. att.
403 +|(% style="width:205px" %)When?|(% style="width:272px" %)data update – last update|(% style="width:178px" %)X|(% style="width:270px" %)time stamp|(% colspan="2" rowspan="1" style="width:478px" %)mandatory attribute
404 +|(% style="width:205px" %)How often?|(% style="width:272px" %)//frequency//|(% style="width:178px" %)X|(% style="width:270px" %)available|(% style="width:290px" %)mand. att. or|(% style="width:221px" %)not relevant
405 +|(% style="width:205px" %)How much?|(% style="width:272px" %)observation value|(% style="width:178px" %)-|(% style="width:270px" %)numeric|(% colspan="2" rowspan="1" style="width:290px" %) measure
432 432  
433 433  ~*~*Concepts in //italics// are only relevant for time series DSDs. An “X” in the COG column means the concept is defined in the COG. Code list “development” means that the SWG will develop a code list to be recommended in the COG; “revision” means that the code list is recommended by the COG and under revision by the SWG; “format” means that a format is defined by another concept; “text”, “time stamp”, and “numeric” provide data types used for uncoded concepts.
434 434  
... ... @@ -436,25 +436,19 @@
436 436  
437 437  **Table 10. Suggested additional concepts for certain scenarios~*~***
438 438  
439 -|**Question**|**Concept**|**COG**|**Code list**|**TS CS**|**Scenario**
413 +|**Question**|**Concept**|**COG**|**Code list**|**TS**|**CS**|**Scenario**
440 440  |Who?|compiling agency|X|development|(((
441 -conditional conditional
442 -
443 - (sibling) (obs. level)
444 -)))|data provider different from data compiler
415 +conditional (sibling)
416 +)))|conditional (obs. level)|data provider different from data compiler
445 445  |Who?|(((
446 -confidentiality
447 -
448 -status – observation
449 -)))|X|available|mandatory (obs. level)|except dissemination
450 -|How?|observation status|X|available|conditional (obs. level)|except orig. collection
418 +confidentiality status – observation
419 +)))|X|available|(% colspan="2" rowspan="1" %)mandatory (obs. level)|except dissemination
420 +|How?|observation status|X|available|(% colspan="2" rowspan="1" %)conditional (obs. level)|except orig. collection
451 451  |How much?|(((
452 -//observation pre-//
422 +//observation pre-break value//
423 +)))|-|numeric|cond. (obs.)|not relevant|except orig. collection
424 +|What and how?|//time series title//|X|text|cond. (TS)|not relevant|dissemination
453 453  
454 -//break value//
455 -)))|-|numeric|cond. (obs.) not relevant|except orig. collection
456 -|What and how?|//time series title//|X|text|cond. (TS) not relevant|dissemination
457 -
458 458  ~** The legend of Table 9 applies to Table 10 as well. The suggested attachment level of attributes (if any) is provided in parentheses in the TS (time series) or CS (cross-section) columns. In case an attribute does not vary at that level in a certain use case, it should be attached at the highest possible level.
459 459  
460 460  == 5.2 Attribute attachment levels and definition of groups ==
... ... @@ -476,10 +476,8 @@
476 476  * //ID//: a unique identifier of the message
477 477  * //Test//: a Boolean attribute that indicates whether the message is for test purposes or not
478 478  * //Prepared//: the date the message was prepared
479 -* //Sender//: the identification of the organization that is transmitting the message
447 +* //Sender//: the identification of the organization that is transmitting the message (recommended: code from the agency code list in the SDMX COG)
480 480  
481 -(recommended: code from the agency code list in the SDMX COG)
482 -
483 483  From a business perspective, the inclusion of the //Name// element is highly recommended, as it can help to understand the purpose of the exchange message. Other header elements such as //Receiver// are optional.
484 484  
485 485  = 6 STEP-BY-STEP GUIDE =
... ... @@ -490,13 +490,15 @@
490 490  
491 491  Figure 1 provides an overview of the overall process. As a first step, the context of the data exchange(s) that should be covered by the DSD(s) is defined in terms of purpose, domains, level of exchange, type of data, type of recipient, role of in data exchange, process pattern, and GSBPM phase (see Figure 2). Since reusing existing artefacts is one of the guiding principles, the second step identifies existing DSDs that may be reused (see Figure 3). In case relevant DSDs are available, their suitability in the present context is evaluated in step 3. Aspects to be taken into account are concept coverage, concept roles, attribute attachment levels, and code lists (see Figure 4). Step 4 is subject to the outcome of step 3. In case of a favorable assessment, the DSDs are simply reused. If the DSDs are partly suitable, modified versions can be derived. See section 2. for a summary of possible DSD modification scenarios. If the DSDs are not suitable or if no relevant DSDs are available at all, new DSDs will be defined as described in section 3. Finally, supporting artefacts such as data flow definitions and data provision agreements are defined (see Figure 5).
492 492  
459 +(% class="wikigeneratedid" %)
460 +[[image:1768470533088-795.png]]
493 493  
494 494  (% class="wikigeneratedid" id="HFigure1.OverviewoftheDSDdesignprocess" %)
495 495  Figure 1. Overview of the DSD design process
496 496  
497 -
498 498  Figure 2 summarizes the characteristics of the data exchange context that is defined in step 1. These characteristics affect the decision on the data structuring approach that is part of the process of defining the concepts of a new DSD (step 4.3. in Figure 1; see Figure 7 in section 2.).
499 499  
467 +[[image:1768470575978-226.png]]
500 500  
501 501  (% class="wikigeneratedid" id="HFigure2.Characteristicsofdataexchangecontext" %)
502 502  Figure 2. Characteristics of data exchange context
... ... @@ -503,20 +503,23 @@
503 503  
504 504  Figure 3 recaps the priorities given to different types of existing DSDs when searching for candidates for reuse in step 2. Global DSDs maintained by the SDMX consortium are ranked the highest. They can be found via the Global SDMX Registry.
505 505  
474 +(% class="wikigeneratedid" %)
475 +[[image:1768470596130-305.png]]
506 506  
507 507  (% class="wikigeneratedid" id="HFigure3.PriorityrankingofexistingDSDsforreuse" %)
508 508  Figure 3. Priority ranking of existing DSDs for reuse
509 509  
510 -
511 511  Figure 4 summarizes the aspects to be considered in the assessment of the suitability of existing DSDs in step 3. For a detailed description of the cases of partial unsuitability see section 2.1. above.
512 512  
482 +(% class="wikigeneratedid" %)
483 +[[image:1768470626558-321.png]]
513 513  
514 514  (% class="wikigeneratedid" id="HFigure4.AspectsofDSDsuitability" %)
515 515  Figure 4. Aspects of DSD suitability
516 516  
517 -
518 518  Figure 5 lists the most relevant artefacts required in addition to a DSD, its concept scheme, and code lists.
519 519  
490 +[[image:1768470646456-652.png]]
520 520  
521 521  Figure 5. Supporting artefacts
522 522  
... ... @@ -524,48 +524,83 @@
524 524  
525 525  Figure 6 briefly recapitulates the actions that can be taken to overcome partial unsuitability of DSDs. As far as possible, existing artefacts should be reused in this case. This means that even if a DSD cannot be reused as a whole, concepts and code lists from that DSD can be included in the new DSD by reference.
526 526  
527 -**Figure 6. DSD modification scenarios**
498 +[[image:1768470678965-391.png]]
528 528  
500 +Figure 6. DSD modification scenarios
501 +
529 529  == 6.3 Defining new DSDs ==
530 530  
531 531  In case no (suitable) DSD is available, the actual process of specifying a new DSD is started. Figure 7 depicts this process (step 4.3. in Figure 1). It encompasses the specification of concepts, code lists, and data formats. All three specification steps include the identification of already existing artefacts that could be reused or modified to satisfy the requirements at hand and the definition of new artefacts in case no suitable artefacts are detected. Several iterations of steps 1 (specification of concepts; see Figure 8) and 2 (specification of code lists; see Figure°13) may be necessary, including revisions of the decision concerning the data structuring approach. Finally all artefacts defined in the previous steps are put together into a DSD.
532 532  
533 -==== Figure 7. New DSD specification process ====
506 +(% class="wikigeneratedid" %)
507 +[[image:1768470705894-724.png]]
534 534  
509 +(% class="wikigeneratedid" id="HFigure7.NewDSDspecificationprocess" %)
510 +Figure 7. New DSD specification process
511 +
535 535  Figure 8 outlines step 4.3.1, the process of concept specification. It covers the decision on the structuring approach, the identification of relevant concepts and the assessment of their suitability, the definition of new concepts, concept roles, and attribute attachment levels.
536 536  
537 -==== Figure 8. Concept specification process ====
514 +(% class="wikigeneratedid" %)
515 +[[image:1768470729899-225.png]]
538 538  
517 +(% class="wikigeneratedid" id="HFigure8.Conceptspecificationprocess" %)
518 +Figure 8. Concept specification process
519 +
539 539  Both, the decision on reuse of existing concepts as well as the definition of new ones, may lead back to a revision of the data structuring approach. For example, it could turn out that a certain concept needs to be broken down further which may lead from a “few composite dimensions” to a “many pure dimensions” approach. Figure 9 provides the design options involved in the decision on a data structuring approach. The options are defined in terms of the number of DSDs and the number of concepts (especially dimensions). The reasonability and feasibility of these options depend on the context of the present data exchange(s) as defined in the first step of the overall design process and on the content of the data exchange with respect to concepts.
540 540  
541 -==== Figure 9. DSD design options ====
522 +(% class="wikigeneratedid" %)
523 +[[image:1768470752201-691.png]]
542 542  
525 +(% class="wikigeneratedid" id="HFigure9.DSDdesignoptions" %)
526 +Figure 9. DSD design options
527 +
543 543  In the second step of new DSD design, relevant existing concepts are identified. Figure 10 indicates potential sources of those concepts such as the SDMX COG for cross-domain concepts, global or other DSDs as already identified earlier in the process, and domain standards such as the UN's System of National Accounts Manual 2008 for domain-specific concepts.
544 544  
545 -==== Figure 10. Potential sources of concepts and definitions ====
530 +(% class="wikigeneratedid" %)
531 +[[image:1768470775109-874.png]]
546 546  
533 +(% class="wikigeneratedid" id="HFigure10.Potentialsourcesofconceptsanddefinitions" %)
534 +Figure 10. Potential sources of concepts and definitions
535 +
547 547  The definition of new concepts (step 4.3.1.4.2.) is necessary if no (suitable) concept can be reused. It entails giving each concept a name, a code, and a definition. Further details about the usage of the concepts in the DSD are specified in steps 4.3.1.5. (concept roles), 4.3.1.6. (dimension groups), and 4.3.1.7. (attribute attachment levels). Figure 11 and 12 summarize the possible concept roles and attribute attachment levels.
548 548  
549 549  The second step in the process of defining a new DSD is the specification of code lists for all coded concepts. All dimensions must be coded (with time being an exception to this rule); attributes may be coded. For uncoded concepts, a data format has to be specified. Existing formats may be reused or new ones defined. An example is the time format that is specified in the SDMX COG. Figure 13 illustrates the code list specification process. If no relevant and suitable code list exists, a new one will be defined or a partially suitable one will be adapted (see Figure 16). Suitable code lists can simply be reused via reference.
550 550  
540 +[[image:1768470796725-270.png]]
551 551  
542 +(% class="wikigeneratedid" %)
543 +Figure 11. Possible concept roles
544 +
545 +(% class="wikigeneratedid" %)
546 +[[image:1768470829131-599.png]]
547 +
548 +(% class="wikigeneratedid" %)
549 +Figure 12. Possible attribute attachment levels
550 +
551 +(% class="wikigeneratedid" %)
552 +[[image:1768470860119-204.png]]
553 +
552 552  (% class="wikigeneratedid" id="HFigure13.Codelistspecificationprocess" %)
553 553  Figure 13. Code list specification process
554 554  
555 -
557 +(% class="wikigeneratedid" %)
556 556  Figure 14 recaps the priorities given to different types of existing code lists when searching for candidates for reuse (step 4.3.2.1.). Code lists recommended by the SDMX COG (and maintained by the SDMX consortium) are ranked the highest.
557 557  
560 +[[image:1768470878394-873.png]]
558 558  
559 559  (% class="wikigeneratedid" id="HFigure14.Priorityrankingofexistingcodelistsforreuse" %)
560 560  Figure 14. Priority ranking of existing code lists for reuse
561 561  
562 -
565 +(% class="wikigeneratedid" %)
563 563  Figure 15 summarizes the aspects to be considered in the evaluation of the suitability of existing code lists (step 4.3.2.2.). Figure 16 summarizes the scenarios of adapting existing code lists that do not fully meet the specified needs (step 4.3.2.3.2). For a detailed description of the cases of partial unsuitability see section 2.1. above.
564 564  
568 +[[image:1768470896763-366.png]]
565 565  
566 566  (% class="wikigeneratedid" id="HFigure15.Aspectsofcodelistsuitability" %)
567 567  Figure 15. Aspects of code list suitability
568 568  
573 +(% class="wikigeneratedid" %)
574 +[[image:1768470911321-123.png]]
569 569  
570 570  (% class="wikigeneratedid" id="HFigure16.Codelistmodificationscenarios" %)
571 571  Figure 16. Code list modification scenarios
... ... @@ -578,8 +578,11 @@
578 578  
579 579  Figure 17 provides an overview of all steps in the DSD design process as described in the previous subsections 1. to 3. Figure 18 compiles those steps into a checklist for DSD designers to help them make sure all aspects are considered.
580 580  
587 +[[image:1768471052577-528.png]]
588 +
581 581  Figure 17. DSD design process
582 582  
591 +[[image:1768470939545-136.png]]
583 583  
584 584  Figure 18. Checklist for DSD design process
585 585  
... ... @@ -629,10 +629,12 @@
629 629  
630 630  == 9.2 Non-SDMX Documents ==
631 631  
632 -6th Edition of the IMF's Balance of Payments Manual (BPM6). Available online at http:~/~/www.imf.org/external/pubs/ft/bop/2007/bopman6.htm.
641 +6th Edition of the IMF's Balance of Payments Manual (BPM6). Available online at [[http:~~/~~/www.imf.org/external/pubs/ft/bop/2007/bopman6.htm>>https://http:www.imf.orgexternalpubsftbop2007bopman6.htm||target="_blank"]].
633 633  
634 -METIS: Generic Statistical Business Process Model (GSBPM). Available online at http:~/~/www1.unece.org/stat/platform/display/metis/The+Generic+Statistical+Business+Process+Model. UN's System of National Accounts Manual 2008 (SNA2008). Available online at http:~/~/unstats.un.org/unsd/nationalaccount/sna2008.asp.
643 +METIS: Generic Statistical Business Process Model (GSBPM). Available online at [[http:~~/~~/www1.unece.org/stat/platform/display/metis/The+Generic+Statistical+Business+Process+Model>>https://http:www1.unece.orgstatplatformdisplaymetisThe+Generic+Statistical+Business+Process+Model||target="_blank"]].
635 635  
645 +UN's System of National Accounts Manual 2008 (SNA2008). Available online at [[http:~~/~~/unstats.un.org/unsd/nationalaccount/sna2008.asp>>https://http:unstats.un.orgunsdnationalaccountsna2008.asp||target="_blank"]].
646 +
636 636  ----
637 637  
638 638  {{putFootnotes/}}
1768470533088-795.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +48.6 KB
Content
1768470575978-226.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +95.8 KB
Content
1768470596130-305.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +44.4 KB
Content
1768470611326-907.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +51.5 KB
Content
1768470626558-321.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +51.5 KB
Content
1768470646456-652.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +32.7 KB
Content
1768470678965-391.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +117.9 KB
Content
1768470705894-724.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +23.9 KB
Content
1768470729899-225.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +66.5 KB
Content
1768470752201-691.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +38.0 KB
Content
1768470775109-874.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +36.5 KB
Content
1768470796725-270.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +30.7 KB
Content
1768470829131-599.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +30.8 KB
Content
1768470860119-204.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +37.9 KB
Content
1768470878394-873.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +49.7 KB
Content
1768470896763-366.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +29.9 KB
Content
1768470911321-123.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +49.2 KB
Content
1768470939545-136.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +143.4 KB
Content
1768471052577-528.png
Author
... ... @@ -1,0 +1,1 @@
1 +xwiki:XWiki.helena
Size
... ... @@ -1,0 +1,1 @@
1 +97.4 KB
Content
© Semantic R&D Group, 2026