Changes for page Guidelines for SDMX Data Structure Definitions

Last modified by Artur K. on 2026/05/29 14:28

From 1.5 to 1.4

From version 1.4

edited by Helena K.
on 2026/01/15 12:22

Change comment: There is no comment for this version

To version 1.1

edited by Helena K.
on 2026/01/15 12:12

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -63,11 +63,9 @@
  In case an existing DSD is close to but differs from what is needed, it may: (i) contain irrelevant concepts, (ii) lack some required concepts, (iii) use the concepts in different roles than required, (iv) deviate with respect to some of the code lists, or (v) contain pure dimensions when mixed dimensions would make more sense or vice versa. More complex situations that are combinations of several (or even all) of these five cases may occur as well. For example, an existing DSD could contain unnecessary concepts and lack other concepts at the same time.
--==== 2.1.3.1 Irrelevant concepts ====
++**2.1.3.1 Irrelevant concepts**
--Two options exist to deal with the situation of only a subset of dimensions being relevant{{footnote}}Technically speaking, a third possibility exists. A structure map can be used to define the reduced DSD. The
--structure map establishes a mapping between a source structure and a target structure. In this special case, the
--aim of the structure map is simply to get rid of irrelevant dimensions. To this end the DSD is mapped to itself, and any unmapped dimensions will not be part of the target structure. The original DSD is not affected by the structure map. The reduced DSD can be derived from the structure map, but from a technical point of view there is no need to actually create the reduced DSD as an artefact. It can exist as a “virtual” DSD that is merely defined by the Structure Map.{{/footnote}}:
++Two options exist to deal with the situation of only a subset of dimensions being relevant[[(% class="wikiinternallink" %)^^~[1~]^^>>path:#_ftn1]](%%):
 . define a new, reduced concept scheme that includes only the relevant concepts and code lists by reference and a new DSD that uses the reduced concept scheme;
 . reuse concept scheme, code lists, and DSD, but add constraints to the data flow definition (or to the DSD, but this would also make it a new, derived DSD) that set the irrelevant dimensions to whatever applies from the following:
@@ -75,7 +75,7 @@
 . If a concept is obsolete because only total values aggregated over the corresponding dimension are relevant, the dimension (or code list) should be restricted to a “total” item. For instance, an existing DSD on bilateral trade contains “Partner Country” as dimension, since data are collected with a breakdown by country of trade counterpart. The new data exchange disseminates similar data, but only the trade totals “vis-à-vis all countries”.
 . If a concept is not needed because it cannot even be relevant for the data at all or because an additional breakdown is just not available, the concept should be restricted to “not applicable” or “unknown” via constraints. For example, a financial instrument breakdown was not collected and it is unclear whether data for all or only for some financial instruments were included, that is whether the “total” value can be used. In this case, the dimension would be restricted to “unknown”. Consider another simple example of a DSD that contains, amongst others, the two dimensions “Unit of Measure” and “Base Period”. The code list for “Unit of Measure” consists of percent per annum, percentage, and index with base year=100. “Base Period” can contain dates, years, months, etc. If the new data exchange restricts “Unit of Measure” to percent per annum, “Base Period” becomes obsolete and should be constrained to “not applicable”.
--==== 2.1.3.2 Missing concepts ====
++**2.1.3.2 Missing concepts**
  In this case additional concepts (and possibly code lists) are required, for example to accommodate an additional cross-classification. One option is to adapt the existing DSD to satisfy the new needs, i.e., create a new version of the DSD by adding the concepts, dimensions/attributes, and code lists. The feasibility of this solution depends on the relation between the organization requiring the new data structure and the organization maintaining the existing DSD, and the relation between the two usage contexts. The original, restricted model needed for the existing data flows can be specified by means of constraints, as described above for irrelevant concepts, or by referring to the original version of the DSD.
@@ -85,22 +85,27 @@
 . A code list similar to what is needed is available somewhere else.
 . If only a subset of the existing code list is relevant, the code list can be reused with a constraint imposed either on the code list, or in the DSD, or in the data flow definition (or in the data provisioning agreement). It is also possible to use the entire code list but only report data for the subset.
 . In case a (different) hierarchy is needed, the underlying flat code list can be referenced and a new hierarchical code list introduced. This means that a flat code list (i.e. without an explicitly defined hierarchy) is available that meets the coverage requirements, but that the existing hierarchy defined on top of the flat code list deviates from the required hierarchy. Hence, the suitable flat code list can be reused, but a new hierarchical code list needs to be defined. Consider for instance the “Reference Area” code list as recommended by the SDMX Content-Oriented Guidelines (COG), i.e. containing ISO-2-character codes for countries. Different groupings of these countries are relevant in different contexts, for example, regional aggregates by continent, by income level, or by membership in certain international groups (e.g. monetary unions). A flat code list can be defined that contains all these country groups in addition to the individual countries. This list does not specify parent-child relationships between groups and countries, as this would entail repeating countries for each group they belong to. It basically provides the value domain for a geographic dimension, but not the semantics of the values in terms of the group composition.
++
  On top of this flat code list, different hierarchical code lists can be defined that may use the complete set of codes or just a subset thereof. The flat code list can be referenced by any DSD with a geographical reference, and different DSDs can build their own hierarchical code lists based on the flat list.
++
++1.
 . If additional items are needed, a derived code list can be specified by including each element from the existing code list by reference and adding the new elements as required. The current versions of the SDMX Technical Standard do not allow combining existing code lists into one or referencing an entire code list and adding a few elements to be managed in the new code list. Often, simply a copy of the existing list is introduced as new code list with the new items included. This is not optimal, as conceptually identical items have to be managed in multiple code lists. At least in theory it is also possible to just create a new version of the existing code list with the additional items. Existing data flows would then either use the original version of the code list or the new version with constraints, whereas the new version of the code list would be used in the new data flow. Again, this option depends on the organizational background.
++
  Consider as an example the inclusion of “Currency” into a DSD with a need for codes for “Domestic currency” and “Foreign currency” in addition to the codes specified in the code list recommended by the SDMX COG. In the first option, the currencies from the recommended code list are included by reference and the two new items added to a new code list. This is superior to the common practice of including copies of the existing codes (the currencies) instead of references. This makes the new code list more independent of the existing one, but it increases the maintenance cost and the risk of inconsistencies. Another option is to extend the existing code list by creating a new code list version. In the currency example, the SDMX consortium as the owner of the recommended code list would need to decide whether this new version should be created or not.
++
 . No appropriate code lists are available. New code lists have to be defined based on the guidelines for the development of code lists. This may often be the case for domain-specific code lists, especially in new areas of investigation.
--==== 2.1.3.3 Concepts in different roles ====
++**2.1.3.3 Concepts in different roles**
  In this case concepts are available in other roles than required, for example what needs to be a dimension is merely an attribute or vice versa. This case is already briefly discussed above as part of the first case (“irrelevant concepts”). Basically, a new DSD has to be defined. It can reuse the concept scheme and code lists, but specifies the concepts in the new DSD as dimensions or attributes as required. In case an attribute needs to become a dimension, it may be necessary to define a new code list for that dimension in case it did not exist previously.
  An example for an attribute having to be redefined as a dimension may be the “Unit of measure” that is frequently just specified as an attribute. If certain indicators are presented in different units in the same data flow, the corresponding DSD must contain “Unit of measure” as dimension, though.
--==== 2.1.3.4 Different code lists ====
++**2.1.3.4 Different code lists**
  In this case the new requirements differ from the existing DSD with respect to some of the code lists, either by only a subset of codes being relevant, by a deviating hierarchical structure, or by necessitating additional codes. These three scenarios are discussed above as special cases of the “missing concepts” case. In theory, just defining a new code list whenever an existing one is not completely appropriate is also possible (but not desirable). However, this means that the overlapping items have to be managed in multiple code lists unless they are included by reference. Also, different DSDs have to be maintained. If the constraints are neither imposed at the code list nor at the DSD level, but at the data flow level, the DSD is simply reused. This is highly recommended. The cost of maintaining multiple DSDs or multiple, largely overlapping code lists can be high. The lack of harmonization has one advantage, though: the increased maintenance and versioning flexibility. For global data exchange, this is not regarded as a reasonable solution.
--==== 2.1.3.5 Pure vs. mixed dimensions ====
++**2.1.3.5 Pure vs. mixed dimensions**
  The design principle of pure dimensions is explained in more detail in subsections 3.3 and section 4 on data structuring approaches. If an existing DSD does not have the desired degree of dimension purity, it is necessary to further decompose and/or combine dimensions of that DSD. This will lead to a new derived DSD and also requires the definition of new (combined or split) code lists, unless they are available from elsewhere.
@@ -128,9 +128,9 @@
  === 2.3.4 Density and sparseness ===
--The //density// of a DSD is closely related to its simplicity whereas //sparseness// often comes along with purity. For a dense DSD, a data flow provides data for all (or the large majority of) cells defined by the Cartesian product[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[2~]^^>>path:#_ftn2]](%%) of the DSD dimensions. This is typically the case for simple DSDs. For pure DSDs with many dimensions, it is usually not feasible to share data 338 for the entire data space created by the combination of all dimensions.
++The //density// of a DSD is closely related to its simplicity whereas //sparseness// often comes along with purity. For a dense DSD, a data flow provides data for all (or the large majority of) cells defined by the Cartesian product[[(% class="wikiinternallink" %)^^~[2~]^^>>path:#_ftn2]](%%) of the DSD dimensions. This is typically the case for simple DSDs. For pure DSDs with many dimensions, it is usually not feasible to share data 338 for the entire data space created by the combination of all dimensions.
--For example, a breakdown by “Institutional Sector” or “Gender” may only make sense for a subset of the “Indicators” provided. The sparseness may be measured in terms of the number of dimensions requiring a “not applicable” value or the number of observations that take at least one “not applicable” or “total” value (both as shares of the total number of dimension or the total number of observations, respectively)[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[3~]^^>>path:#_ftn3]](%%). An even more precise measure of sparseness is the proportion of theoretically possible key combinations that are irrelevant or not feasible or do not carry data.
++For example, a breakdown by “Institutional Sector” or “Gender” may only make sense for a subset of the “Indicators” provided. The sparseness may be measured in terms of the number of dimensions requiring a “not applicable” value or the number of observations that take at least one “not applicable” or “total” value (both as shares of the total number of dimension or the total number of observations, respectively)[[(% class="wikiinternallink" %)^^~[3~]^^>>path:#_ftn3]](%%). An even more precise measure of sparseness is the proportion of theoretically possible key combinations that are irrelevant or not feasible or do not carry data.
  === 2.3.5 Unambiguousness ===
@@ -189,7 +189,7 @@
  The global BOP DSD that is currently being developed may serve as a more specific example for a multi-purpose DSD. It is supposed to support, amongst others, exchange of the ECB's Balance of Payments (BOP) and International Reserves Template (IRT) data, Eurostat's International Investment Position (IIP) and Trade in Services (TS) data, the OECD's BOP data, and the IMF's Coordinated Portfolio Investment (CPIS) and Coordinated Direct Investment (CDIS) data.
--Table 3 below shows some of the concepts considered relevant for some or all of these related data exchange exercises.[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[4~]^^>>path:#_ftn4]](%%) Reporting Country and Unit of Measure are required by all data exchanges; the other concepts listed are only necessary (marked by an “X”) for a subset of the data exchanges. For instance, Eurostat's TS and IMF’s CDIS data do not require the distinction of flows and stocks, different maturities, or valuations (indicated by an “O”). Still, there is value in defining one master DSD that covers all concepts required for all of the data exchanges.
++Table 3 below shows some of the concepts considered relevant for some or all of these related data exchange exercises.[[(% class="wikiinternallink" %)^^~[4~]^^>>path:#_ftn4]](%%) Reporting Country and Unit of Measure are required by all data exchanges; the other concepts listed are only necessary (marked by an “X”) for a subset of the data exchanges. For instance, Eurostat's TS and IMF’s CDIS data do not require the distinction of flows and stocks, different maturities, or valuations (indicated by an “O”). Still, there is value in defining one master DSD that covers all concepts required for all of the data exchanges.
  If that approach is pursued, satellite DSDs for the individual purposes (or exchange exercises) can be created via constraints (and/or structure maps). Each exchange exercise may also be represented as a data flow (the constraints may also be defined in the data flow instead of the DSD). So there would be one data flow defined for each column in the table below. For instance, the IMF CPIS data flow would restrict “Flows and stocks indicator” and “Valuation” to certain values from the respective code lists. Data provision agreements may then be set up for each data flow with each reporting country. Constraints can be used to restrict the contribution of each country to its own data, so “Reporting country” would be set to the respective value. If the constraints are defined in the data flow and/or structure maps are used to exclude irrelevant dimensions, the satellite DSDs do not materialize; they are “virtual” DSDs.
@@ -197,13 +197,13 @@
  |(% rowspan="2" %)**Concept**|(% colspan="2" %)**ECB**|(% colspan="2" %)**Eurostat**|**OECD**|(% colspan="2" %)**IMF**
  |**IRT**|**BOP**|**IIP**|**TS**|**BOP**|**CPIS**|**CDIS**
--|Reporting country or area|X|X|X|X|X|X|X
--|Unit of measure|X|X|X|X|X|X|X
--|Flows and stocks indicator|X|O|O|O|O|O|O
--|Reporting sector|X|X|X|O|X|X|X
--|Financial instrument|X|X|X|O|X|X|X
--|Maturity|X|X|X|O|X|X|O
--|Valuation|X|O|X|O|O|O|O
++|**Reporting country or area**|X|X|X|X|X|X|X
++|**Unit of measure**|X|X|X|X|X|X|X
++|**Flows and stocks indicator**|X|O|O|O|O|O|O
++|**Reporting sector**|X|X|X|O|X|X|X
++|**Financial instrument**|X|X|X|O|X|X|X
++|**Maturity**|X|X|X|O|X|X|O
++|**Valuation**|X|O|X|O|O|O|O
  == 3.4 Type of data exchange and recipient ==
@@ -406,7 +406,7 @@
  = 5 MINIMUM STRUCTURAL AND SEMANTIC REQUIREMENTS =
--Although each data exchange scenario has specific requirements, especially on whether a concept needs to be a dimension, a mandatory or conditional attribute, on the attachment level of attributes, and on the attributes provided in the header of a DSD, a small set of minimum structural and semantic requirements can be defined for all scenarios.[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[5~]^^>>path:#_ftn5]]
++Although each data exchange scenario has specific requirements, especially on whether a concept needs to be a dimension, a mandatory or conditional attribute, on the attachment level of attributes, and on the attributes provided in the header of a DSD, a small set of minimum structural and semantic requirements can be defined for all scenarios.[[(% class="wikiinternallink" %)^^~[5~]^^>>path:#_ftn5]]
  Certain concepts can be broadly agreed upon as being relevant in any data exchange, although their roles may differ between scenarios. The SDMX Content-Oriented Guidelines define many of these cross-domain concepts and, thus, should be referred to for further details on their specification.
@@ -556,22 +556,18 @@
  The second step in the process of defining a new DSD is the specification of code lists for all coded concepts. All dimensions must be coded (with time being an exception to this rule); attributes may be coded. For uncoded concepts, a data format has to be specified. Existing formats may be reused or new ones defined. An example is the time format that is specified in the SDMX COG. Figure 13 illustrates the code list specification process. If no relevant and suitable code list exists, a new one will be defined or a partially suitable one will be adapted (see Figure 16). Suitable code lists can simply be reused via reference.
--(% class="wikigeneratedid" id="HFigure13.Codelistspecificationprocess" %)
--Figure 13. Code list specification process
++==== Figure 13. Code list specification process ====
  Figure 14 recaps the priorities given to different types of existing code lists when searching for candidates for reuse (step 4.3.2.1.). Code lists recommended by the SDMX COG (and maintained by the SDMX consortium) are ranked the highest.
--(% class="wikigeneratedid" id="HFigure14.Priorityrankingofexistingcodelistsforreuse" %)
--Figure 14. Priority ranking of existing code lists for reuse
++==== Figure 14. Priority ranking of existing code lists for reuse ====
  Figure 15 summarizes the aspects to be considered in the evaluation of the suitability of existing code lists (step 4.3.2.2.). Figure 16 summarizes the scenarios of adapting existing code lists that do not fully meet the specified needs (step 4.3.2.3.2). For a detailed description of the cases of partial unsuitability see section 2.1. above.
--(% class="wikigeneratedid" id="HFigure15.Aspectsofcodelistsuitability" %)
--Figure 15. Aspects of code list suitability
++==== Figure 15. Aspects of code list suitability ====
--(% class="wikigeneratedid" id="HFigure16.Codelistmodificationscenarios" %)
--Figure 16. Code list modification scenarios
++==== Figure 16. Code list modification scenarios ====
  If new code lists need to be defined, please refer to the SDMX Guidelines for the creation of code lists. Basically, code, name, and definition need to be provided for each item in the code list. In addition, hierarchical code lists may be defined.
@@ -581,10 +581,10 @@
  Figure 17 provides an overview of all steps in the DSD design process as described in the previous subsections 1. to 3. Figure 18 compiles those steps into a checklist for DSD designers to help them make sure all aspects are considered.
--Figure 17. DSD design process
++**Figure 17. DSD design process**
--Figure 18. Checklist for DSD design process
++**Figure 18. Checklist for DSD design process**
  = 7 ANNEX 1. GLOSSARY OF TERMS =
@@ -596,7 +596,7 @@
  Concepts assume different roles in a data structure definition:
--* //dimensions// are required to uniquely identify an observation (a data value); e.g., for time series, at least one geographic, one temporal, and one (“mixed") subject-matter dimension are required to identify a data value (for instance: reference area = Mexico, time = 2002, indicator = GDP nominal, US$)[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[6~]^^>>path:#_ftn6]](%%);
++* //dimensions// are required to uniquely identify an observation (a data value); e.g., for time series, at least one geographic, one temporal, and one (“mixed") subject-matter dimension are required to identify a data value (for instance: reference area = Mexico, time = 2002, indicator = GDP nominal, US$)[[(% class="wikiinternallink" %)^^~[6~]^^>>path:#_ftn6]](%%);
  * //measures// are the containers of the actual observation or data values;
  * //attributes// provide additional meta-information required to interpret the data correctly but not to identify the observations; for instance, data for the same observation defined by a value combination of the dimensions (also termed “key”) will usually only be provided for one unit multiplier, e.g. in millions; hence unit multiplier is not necessary to identify an observation, but it is still required for a proper interpretation. Attributes can be defined as mandatory or not mandatory, and they can be attached at different levels, e.g. at observation level or at the level of groups defined by the value combinations of a predefined subset of dimensions (for example reporting currency may be attached at the country level).
@@ -639,6 +639,8 @@
  ----
++[[~[1~]>>path:#_ftnref1]] Technically speaking, a third possibility exists. A structure map can be used to define the reduced DSD. The structure map establishes a mapping between a source structure and a target structure. In this special case, the aim of the structure map is simply to get rid of irrelevant dimensions. To this end the DSD is mapped to itself, and any unmapped dimensions will not be part of the target structure. The original DSD is not affected by the structure map. The reduced DSD can be derived from the structure map, but from a technical point of view there is no need to actually create the reduced DSD as an artefact. It can exist as a “virtual” DSD that is merely defined by the Structure Map.
++
  [[~[2~]>>path:#_ftnref2]] A Cartesian product (or product set) is a mathematical construct that builds a new set out of a number of given sets. Each member of the Cartesian product corresponds to the selection of one element each in every one of the original sets.
  [[~[3~]>>path:#_ftnref3]] In case a structure map is used to define reduced versions of the DSD, the number of unmapped dimensions is the equivalent measure of sparseness.
@@ -648,7 +648,3 @@
  [[~[5~]>>path:#_ftnref5]] For other more technical requirements such as the admissible characters in a code or label see the SDMX technical documents.
  [[~[6~]>>path:#_ftnref6]] Please note that this is not a recommendation to always have three dimensions only. This is just a simplified example.
--
------
--
--{{putFootnotes/}}