Changes for page Guidelines for SDMX Data Structure Definitions

Last modified by Artur K. on 2026/05/29 14:28

From 1.5 to 1.4 From 1.11 to 1.10

From version 1.10

edited by Helena K.
on 2026/01/15 12:40

Change comment: There is no comment for this version

To version 1.5

edited by Helena K.
on 2026/01/15 12:30

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)
Attachments (0 modified, 0 added, 1 removed)
- 1768469652632-803.png

Details

Page properties

Content

@@ -245,31 +245,50 @@
  The range of options between the “//just one//”// //(mixed) and “//all component//” subject-matter dimensions approaches is subject to the comprehensiveness (i.e. size, coverage) of the data exchange that the DSD is being developed for. If using a “mixed dimensions” approach, rules for the composition of the mixed dimension(s) may be specified (e.g. concatenate concepts A, B, and C to get mixed dimension X), allowing their easy re-decomposition. In general composite dimensions should be avoided as previously recommended by the SDMX Technical Notes, but there are cases that suggest the usage of composite dimensions. Table 4 juxtaposes general pros and cons of the “//many pure concepts//” and “//fewer composite concepts//” approaches.
--**Table 4. General comparison of data structuring approaches**
++|**Many pure concepts**|**Few composite concepts**
++|cleaner data structure|(((
++Mixed dimensions may be composed inconsistently making the decomposition into purer concepts and code lists difficult
--|(% style="width:360px" %)**Many pure concepts**|(% style="width:1255px" %)**Few composite concepts**
--|(% style="width:360px" %)cleaner data structure|(% style="width:1255px" %)(((
--Mixed dimensions may be composed inconsistently making the decomposition into purer concepts and code lists difficult (requiring complex mapping etc.). Information that corresponds to the same concept may be included in different dimensions, e.g. reference year is contained in the indicator dimension in the first example but in the unit in the second example below. The optimal common data structure would consist of Economic Indicator, Unit, and Base period.
++(requiring complex mapping etc.). Information that corresponds to the same concept may be included in different dimensions, e.g. reference year is contained in the indicator dimension in the first example but in the unit in the second example below. The optimal common data structure would consist of Economic Indicator, Unit, and Base period.
++)))
--[[image:1768469652632-803.png||height="106" width="352"]]
++**Table 4. General comparison of data structuring approaches**
++|(% rowspan="3" %)(((
--)))
--|(% style="width:360px" %)shorter and simpler code lists|(% style="width:1255px" %)code lists longer and more complex, may require hierarchy to be “readable”
--|(% style="width:360px" %)more flexible in terms of defining constraints, but constraints more complex|(% style="width:1255px" %)simpler constraints, but some constraints may be difficult to be represented because of mixed dimensions. Consider for instance a constraint “Base period = 1995” in the above example, where some observations include the base period in the Economic Indicator dimension, others in the Unit dimension. Instead of specifying a constraint on a pure Base Period dimension, the constraints may have to be specified at observation (or time series) level
--|(% style="width:360px" %)more flexible in terms of mapping to other data structures (used by other systems), further processing and analysis (e.g. tabulation, dissemination format), and future needs|(% style="width:1255px" %)“mixed” dimensions make data structure less flexible in these respects
--|(% style="width:360px" %)longer (i.e. more complex) observation keys|(% style="width:1255px" %)shorter keys
--|(% style="width:360px" %)special values of code lists such as “not applicable”, “total” may be rather heavily used|(% style="width:1255px" %)less usage of these special values
--|(% style="width:360px" %)creates sparse data if many observations use “not applicable”|(% style="width:1255px" %)way to avoid sparseness
--|(% style="width:360px" %)many constraints may be necessary due to sparseness|(% style="width:1255px" %)typically fewer constraints required because data are less sparse
--|(% style="width:360px" %)many dimensions are tantamount to many attachment levels for attributes (i.e. DSD more flexible in terms of attribute attachment)|(% style="width:1255px" %)less dimensions = less possible attribute attachment levels
--|(% style="width:360px" %)more difficult to handle by an end user|(% style="width:1255px" %)presumably more easily comprehensible and manageable by an end user
--|(% style="width:360px" %)more flexible in terms of defining queries; can be mapped to any “mixed” representation|(% style="width:1255px" %)less flexible in terms of search and retrieval
++
++)))|**Economic Indicator**|**Unit**
++|Industrial production (2000=100)|Index
++|GDP real|US Dollars at 2005 prices
++
++shorter and simpler code lists                               code lists longer and more complex, may
++
++require hierarchy to be “readable”
++
++
++**Many pure concepts                                          Few composite concepts**
++
++more flexible in terms of defining constraints,     simpler constraints, but some constraints may
++
++but constraints more complex                          be difficult to be represented because of mixed
++
++dimensions. Consider for instance a constraint “Base period = 1995” in the above example, where some observations include the base period in the Economic Indicator dimension, others in the Unit dimension. Instead of specifying a constraint on a pure Base Period dimension, the constraints may have to be specified at observation (or time series) level
++
++|more flexible in terms of mapping to other data structures (used by other systems), further processing and analysis (e.g. tabulation, dissemination format), and future needs|“mixed” dimensions make data structure less flexible in these respects
++|longer (i.e. more complex) observation keys|shorter keys
++|special values of code lists such as “not applicable”, “total” may be rather heavily used|less usage of these special values
++|creates sparse data if many observations use “not applicable”|way to avoid sparseness
++|many constraints may be necessary due to sparseness|typically fewer constraints required because data are less sparse
++|many dimensions are tantamount to many attachment levels for attributes (i.e. DSD more flexible in terms of attribute attachment)|less dimensions = less possible attribute attachment levels
++|more difficult to handle by an end user|presumably more easily comprehensible and manageable by an end user
++|more flexible in terms of defining queries; can be mapped to any “mixed” representation|less flexible in terms of search and retrieval
++
++
++
  The latter two aspects mentioned in the table could be summarized as the “many pure dimensions” approach being more difficult to handle for a “basic” user, but providing fewer options for an “advanced” user. When it comes to dissemination to end users, a purer data structure is the appropriate format for consumption by applications and advanced users. For less advanced user groups it makes sense to hide the (for them: unnecessary) complexity by means of concatenating dimensions, for instance to create a time series view.
--Comparing single-purpose and single-domain exchange scenarios with multi-domain and/or multi-purpose scenarios, pure concepts are typically easier to achieve in the former, whereas composite concepts/dimensions may make life easier in the latter, especially because certain cross-classification concepts may only apply to some domains and/or purposes covered. “Purpose” mean“mixed” dimensions make data structure less
--flexible in these respectsither a certain data exchange exercise or data flow, for instance in the BOP DSD endeavor mentioned above each column represents one “purpose”, e.g. ECB IRT or OECD BOP. In multi-domain or –purpose scenarios, pure concepts are more easily obtained by a “many DSDs” approach, no matter if those are independent from each other or linked by a “master DSD”. Although it does not rule out the specification of pure concepts, a “one DSD” approach typically leads to using fewer, composite concepts (dimensions) in those scenarios.
++Comparing single-purpose and single-domain exchange scenarios with multi-domain and/or multi-purpose scenarios, pure concepts are typically easier to achieve in the former, whereas composite concepts/dimensions may make life easier in the latter, especially because certain cross-classification concepts may only apply to some domains and/or purposes covered. “Purpose” means either a certain data exchange exercise or data flow, for instance in the BOP DSD endeavor mentioned above each column represents one “purpose”, e.g. ECB IRT or OECD BOP. In multi-domain or –purpose scenarios, pure concepts are more easily obtained by a “many DSDs” approach, no matter if those are independent from each other or linked by a “master DSD”. Although it does not rule out the specification of pure concepts, a “one DSD” approach typically leads to using fewer, composite concepts (dimensions) in those scenarios.
  Table 5 provides an overview of the pros and cons of the “many pure concepts" and “fewer composite concepts” approaches in different data exchange settings with respect to the type of organizations involved. In any of these settings it is always possible to use one of the data structures that may already exist at one of the involved parties as DSD for the data exchange. The benefits and drawbacks discussed in the table assume that a new DSD is to be defined. A distinction between two different types of intended recipients is implicitly made. Inter-organizational data exchange is mostly machine-to-machine, whereas dissemination of data to end-users is often machine-to-user.
@@ -278,13 +278,18 @@
  |**Level of data exchange**|**Pure vs. composite concepts approach**
  |**within an organization**|(((
  Depends on diversity of systems involved in data exchange.
++
  The approach that requires the least mapping (and similar processing) steps between the two communicating data structures is preferable in terms of a “quick win” solution.
++
  In general, a more granular model is preferable due to its flexibility that helps support potential future needs (with respect to processing, analysis, exchange, dissemination, etc.).
++
  However, an internal exchange should not be made more complex than necessary. If the structures of the communicating systems are comparable, it may not make sense to create an artificial intermediary structure that is more pure, but also more complex than both underlying structures.
++
  Still, as a longer-term strategy it seems reasonable to define a set of internal “standard” code lists that all systems can map to. This allows bilateral communication via the shared concepts and code lists meaning that every data structure only has to be mapped once – to the internal standard – to be able to communicate with all other participating (i.e. mapped) systems.
  )))
  |**between organizations at national level**|(((
  The pros and cons at this level of exchange are comparable to those at the “within organization” level. If the data structures of the communicating systems are comparable, there is no need to introduce complexity by a conceptually optimal, pure data structure. However, if the data structures deviate to a greater extent (and they often do), they should both be decomposed to find a “common denominator”, a more granular “exchange vocabulary” which they can be mapped to.
++
  If related international or national standards exist, they should be used, even though national labels and/or additional levels of detail may be required in the code lists.
  )))
  |**between international organization and national organizations of member countries**|International organizations should collect data at a level of granularity and purity that is most suitable for the intended (and potential future) analyses. The tradeoff with the higher complexity of constraints required to check structural validity of collected data needs to be taken into account as well. Also it is recommended to consider the burden that a more complex data structure may put on national data providers. However, once a DSD is defined, its lifetime is expected to be a number of years. The main effort of the data provider is to specify the mapping from the production data structure to the DSD. Once this is done the data exchange can be automated and the complexity of the DSD does not matter that much.
@@ -359,6 +359,7 @@
  in both cases: important to include only concepts, code lists, and codes actually available / used by the data
  )))
++| | | | |
  In general, finding the “perfect” data structure is less important for bilateral data exchange. Independent, custom-tailored DSDs may do the job quite well, as harmonization and standardization are typically not of high importance. If the data exchange is just a part of a more comprehensive scenario (e.g. multi-purpose, multi-domain, gateway, or data-sharing scenarios), a master DSD with satellite DSDs is preferable.
@@ -452,8 +452,10 @@
  == 5.2 Attribute attachment levels and definition of groups ==
--Each concept can only be used once as a dimension or an attribute in one DSD. Each attribute must be explicitly attached to an observation, series, or group. The attachment level depends on whether the value of the attribute changes by observation, observation group, or time series, or is the same for all observations. In the latter case, the attribute has to be specified at the //data flow// or //dataset// level. For some attributes described in the previous section, a certain attachment level applies, for others the attachment level depends on the data. For example, the time series title has to be attached at the time series level and the observation status at the observation level.
++Each concept can only be used once as a dimension or an attribute in one DSD. Each attribute must be explicitly attached to an observation, series, or group. The attachment level
++depends on whether the value of the attribute changes by observation, observation group, or time series, or is the same for all observations. In the latter case, the attribute has to be specified at the //data flow// or //dataset// level. For some attributes described in the previous section, a certain attachment level applies, for others the attachment level depends on the data. For example, the time series title has to be attached at the time series level and the observation status at the observation level.
++
  Series and groups are useful groupings of data that allow the specification of attributes for a set of observations instead of having to declare those attributes for every data point thereby. This increases the readability of an SDMX data file, reduces the size of the data file, and (in some cases) even increases the processing efficiency.
  Series is relevant for time series data only. It refers to a group of observations that differ only with respect to the time dimension, i.e. all dimensions except time define the series attachment level. The best-known example of a group definition is the sibling group that combines time series with different frequencies. Observations in a sibling group differ with respect to frequency and time; all other dimensions are used to define the sibling group. A sibling group can be regarded as a time series group with the frequency excluded from the group definition. Any other combination of dimensions (or a single dimension) can also be used to define an observation group. An example for a group defined by a single dimension is reporting country. For instance, attributes related to methodology are often the same for all data of a country. In order to attach attributes to a group, a name for that group has to be specified.

1768469652632-803.png

Author

...	...	@@ -1,1 +1,0 @@
1		-xwiki:XWiki.helena

Size

...	...	@@ -1,1 +1,0 @@
1		-10.7 KB

Content