Changes for page Guidelines for SDMX Data Structure Definitions

Last modified by Artur K. on 2026/05/29 14:28

From 1.4 to 1.3 From 1.8 to 1.7

From version 1.7

edited by Helena K.
on 2026/01/15 12:31

Change comment: There is no comment for this version

To version 1.4

edited by Helena K.
on 2026/01/15 12:22

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)
Attachments (0 modified, 0 added, 2 removed)
- 1768468989793-901.png
- 1768469016538-287.png

Details

Page properties

Content

@@ -128,9 +128,9 @@
  === 2.3.4 Density and sparseness ===
--The //density// of a DSD is closely related to its simplicity whereas //sparseness// often comes along with purity. For a dense DSD, a data flow provides data for all (or the large majority of) cells defined by the Cartesian product{{footnote}}A Cartesian product (or product set) is a mathematical construct that builds a new set out of a number of given sets. Each member of the Cartesian product corresponds to the selection of one element each in every one of the original sets.{{/footnote}} of the DSD dimensions. This is typically the case for simple DSDs. For pure DSDs with many dimensions, it is usually not feasible to share data 338 for the entire data space created by the combination of all dimensions.
++The //density// of a DSD is closely related to its simplicity whereas //sparseness// often comes along with purity. For a dense DSD, a data flow provides data for all (or the large majority of) cells defined by the Cartesian product[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[2~]^^>>path:#_ftn2]](%%) of the DSD dimensions. This is typically the case for simple DSDs. For pure DSDs with many dimensions, it is usually not feasible to share data 338 for the entire data space created by the combination of all dimensions.
--For example, a breakdown by “Institutional Sector” or “Gender” may only make sense for a subset of the “Indicators” provided. The sparseness may be measured in terms of the number of dimensions requiring a “not applicable” value or the number of observations that take at least one “not applicable” or “total” value (both as shares of the total number of dimension or the total number of observations, respectively){{footnote}}In case a structure map is used to define reduced versions of the DSD, the number of unmapped dimensions is the equivalent measure of sparseness.{{/footnote}}. An even more precise measure of sparseness is the proportion of theoretically possible key combinations that are irrelevant or not feasible or do not carry data.
++For example, a breakdown by “Institutional Sector” or “Gender” may only make sense for a subset of the “Indicators” provided. The sparseness may be measured in terms of the number of dimensions requiring a “not applicable” value or the number of observations that take at least one “not applicable” or “total” value (both as shares of the total number of dimension or the total number of observations, respectively)[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[3~]^^>>path:#_ftn3]](%%). An even more precise measure of sparseness is the proportion of theoretically possible key combinations that are irrelevant or not feasible or do not carry data.
  === 2.3.5 Unambiguousness ===
@@ -138,8 +138,6 @@
  **Table 1. Unambiguousness example – dimensions**
--[[image:1768469016538-287.png]]
--
  How would an observation of “Gross domestic product, volume, US dollars, reference year = 2005, millions” for the United States be represented with these dimensions? Table 2 provides three different possible representations (there may be even more).
  **Table 2. Unambiguousness example – ambiguous representations**
@@ -191,7 +191,7 @@
  The global BOP DSD that is currently being developed may serve as a more specific example for a multi-purpose DSD. It is supposed to support, amongst others, exchange of the ECB's Balance of Payments (BOP) and International Reserves Template (IRT) data, Eurostat's International Investment Position (IIP) and Trade in Services (TS) data, the OECD's BOP data, and the IMF's Coordinated Portfolio Investment (CPIS) and Coordinated Direct Investment (CDIS) data.
--Table 3 below shows some of the concepts considered relevant for some or all of these related data exchange exercises.{{footnote}}Please note that the example is taken from the development status of the BOP DSD at the time of writing this document. The concepts and their relevance for certain data exchanges (represented as data flows or derived DSDs) may be different in the final version of the DSD.{{/footnote}} Reporting Country and Unit of Measure are required by all data exchanges; the other concepts listed are only necessary (marked by an “X”) for a subset of the data exchanges. For instance, Eurostat's TS and IMF’s CDIS data do not require the distinction of flows and stocks, different maturities, or valuations (indicated by an “O”). Still, there is value in defining one master DSD that covers all concepts required for all of the data exchanges.
++Table 3 below shows some of the concepts considered relevant for some or all of these related data exchange exercises.[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[4~]^^>>path:#_ftn4]](%%) Reporting Country and Unit of Measure are required by all data exchanges; the other concepts listed are only necessary (marked by an “X”) for a subset of the data exchanges. For instance, Eurostat's TS and IMF’s CDIS data do not require the distinction of flows and stocks, different maturities, or valuations (indicated by an “O”). Still, there is value in defining one master DSD that covers all concepts required for all of the data exchanges.
  If that approach is pursued, satellite DSDs for the individual purposes (or exchange exercises) can be created via constraints (and/or structure maps). Each exchange exercise may also be represented as a data flow (the constraints may also be defined in the data flow instead of the DSD). So there would be one data flow defined for each column in the table below. For instance, the IMF CPIS data flow would restrict “Flows and stocks indicator” and “Valuation” to certain values from the respective code lists. Data provision agreements may then be set up for each data flow with each reporting country. Constraints can be used to restrict the contribution of each country to its own data, so “Reporting country” would be set to the respective value. If the constraints are defined in the data flow and/or structure maps are used to exclude irrelevant dimensions, the satellite DSDs do not materialize; they are “virtual” DSDs.
@@ -383,6 +383,7 @@
  in both cases: important to include only concepts, code lists, and codes actually available / used by the data
  )))
++| | | | |
  In general, finding the “perfect” data structure is less important for bilateral data exchange. Independent, custom-tailored DSDs may do the job quite well, as harmonization and standardization are typically not of high importance. If the data exchange is just a part of a more comprehensive scenario (e.g. multi-purpose, multi-domain, gateway, or data-sharing scenarios), a master DSD with satellite DSDs is preferable.
@@ -407,7 +407,7 @@
  = 5 MINIMUM STRUCTURAL AND SEMANTIC REQUIREMENTS =
--Although each data exchange scenario has specific requirements, especially on whether a concept needs to be a dimension, a mandatory or conditional attribute, on the attachment level of attributes, and on the attributes provided in the header of a DSD, a small set of minimum structural and semantic requirements can be defined for all scenarios.{{footnote}}For other more technical requirements such as the admissible characters in a code or label see the SDMX technical documents.{{/footnote}}
++Although each data exchange scenario has specific requirements, especially on whether a concept needs to be a dimension, a mandatory or conditional attribute, on the attachment level of attributes, and on the attributes provided in the header of a DSD, a small set of minimum structural and semantic requirements can be defined for all scenarios.[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[5~]^^>>path:#_ftn5]]
  Certain concepts can be broadly agreed upon as being relevant in any data exchange, although their roles may differ between scenarios. The SDMX Content-Oriented Guidelines define many of these cross-domain concepts and, thus, should be referred to for further details on their specification.
@@ -476,8 +476,10 @@
  == 5.2 Attribute attachment levels and definition of groups ==
--Each concept can only be used once as a dimension or an attribute in one DSD. Each attribute must be explicitly attached to an observation, series, or group. The attachment level depends on whether the value of the attribute changes by observation, observation group, or time series, or is the same for all observations. In the latter case, the attribute has to be specified at the //data flow// or //dataset// level. For some attributes described in the previous section, a certain attachment level applies, for others the attachment level depends on the data. For example, the time series title has to be attached at the time series level and the observation status at the observation level.
++Each concept can only be used once as a dimension or an attribute in one DSD. Each attribute must be explicitly attached to an observation, series, or group. The attachment level
++depends on whether the value of the attribute changes by observation, observation group, or time series, or is the same for all observations. In the latter case, the attribute has to be specified at the //data flow// or //dataset// level. For some attributes described in the previous section, a certain attachment level applies, for others the attachment level depends on the data. For example, the time series title has to be attached at the time series level and the observation status at the observation level.
++
  Series and groups are useful groupings of data that allow the specification of attributes for a set of observations instead of having to declare those attributes for every data point thereby. This increases the readability of an SDMX data file, reduces the size of the data file, and (in some cases) even increases the processing efficiency.
  Series is relevant for time series data only. It refers to a group of observations that differ only with respect to the time dimension, i.e. all dimensions except time define the series attachment level. The best-known example of a group definition is the sibling group that combines time series with different frequencies. Observations in a sibling group differ with respect to frequency and time; all other dimensions are used to define the sibling group. A sibling group can be regarded as a time series group with the frequency excluded from the group definition. Any other combination of dimensions (or a single dimension) can also be used to define an observation group. An example for a group defined by a single dimension is reporting country. For instance, attributes related to methodology are often the same for all data of a country. In order to attach attributes to a group, a name for that group has to be specified.
@@ -507,36 +507,25 @@
  Figure 1 provides an overview of the overall process. As a first step, the context of the data exchange(s) that should be covered by the DSD(s) is defined in terms of purpose, domains, level of exchange, type of data, type of recipient, role of in data exchange, process pattern, and GSBPM phase (see Figure 2). Since reusing existing artefacts is one of the guiding principles, the second step identifies existing DSDs that may be reused (see Figure 3). In case relevant DSDs are available, their suitability in the present context is evaluated in step 3. Aspects to be taken into account are concept coverage, concept roles, attribute attachment levels, and code lists (see Figure 4). Step 4 is subject to the outcome of step 3. In case of a favorable assessment, the DSDs are simply reused. If the DSDs are partly suitable, modified versions can be derived. See section 2. for a summary of possible DSD modification scenarios. If the DSDs are not suitable or if no relevant DSDs are available at all, new DSDs will be defined as described in section 3. Finally, supporting artefacts such as data flow definitions and data provision agreements are defined (see Figure 5).
++==== Figure 1. Overview of the DSD design process ====
--(% class="wikigeneratedid" id="HFigure1.OverviewoftheDSDdesignprocess" %)
--Figure 1. Overview of the DSD design process
--
--
  Figure 2 summarizes the characteristics of the data exchange context that is defined in step 1. These characteristics affect the decision on the data structuring approach that is part of the process of defining the concepts of a new DSD (step 4.3. in Figure 1; see Figure 7 in section 2.).
--(% class="wikigeneratedid" id="HFigure2.Characteristicsofdataexchangecontext" %)
--Figure 2. Characteristics of data exchange context
++==== Figure 2. Characteristics of data exchange context ====
  Figure 3 recaps the priorities given to different types of existing DSDs when searching for candidates for reuse in step 2. Global DSDs maintained by the SDMX consortium are ranked the highest. They can be found via the Global SDMX Registry.
++==== Figure 3. Priority ranking of existing DSDs for reuse ====
--(% class="wikigeneratedid" id="HFigure3.PriorityrankingofexistingDSDsforreuse" %)
--Figure 3. Priority ranking of existing DSDs for reuse
--
--
  Figure 4 summarizes the aspects to be considered in the assessment of the suitability of existing DSDs in step 3. For a detailed description of the cases of partial unsuitability see section 2.1. above.
++==== Figure 4. Aspects of DSD suitability ====
--(% class="wikigeneratedid" id="HFigure4.AspectsofDSDsuitability" %)
--Figure 4. Aspects of DSD suitability
--
--
  Figure 5 lists the most relevant artefacts required in addition to a DSD, its concept scheme, and code lists.
++**Figure 5. Supporting artefacts**
--Figure 5. Supporting artefacts
--
  == 6.2 Defining modified DSDs ==
  Figure 6 briefly recapitulates the actions that can be taken to overcome partial unsuitability of DSDs. As far as possible, existing artefacts should be reused in this case. This means that even if a DSD cannot be reused as a whole, concepts and code lists from that DSD can be included in the new DSD by reference.
@@ -569,14 +569,11 @@
  (% class="wikigeneratedid" id="HFigure13.Codelistspecificationprocess" %)
  Figure 13. Code list specification process
--
  Figure 14 recaps the priorities given to different types of existing code lists when searching for candidates for reuse (step 4.3.2.1.). Code lists recommended by the SDMX COG (and maintained by the SDMX consortium) are ranked the highest.
--
  (% class="wikigeneratedid" id="HFigure14.Priorityrankingofexistingcodelistsforreuse" %)
  Figure 14. Priority ranking of existing code lists for reuse
--
  Figure 15 summarizes the aspects to be considered in the evaluation of the suitability of existing code lists (step 4.3.2.2.). Figure 16 summarizes the scenarios of adapting existing code lists that do not fully meet the specified needs (step 4.3.2.3.2). For a detailed description of the cases of partial unsuitability see section 2.1. above.
@@ -583,7 +583,6 @@
  (% class="wikigeneratedid" id="HFigure15.Aspectsofcodelistsuitability" %)
  Figure 15. Aspects of code list suitability
--
  (% class="wikigeneratedid" id="HFigure16.Codelistmodificationscenarios" %)
  Figure 16. Code list modification scenarios
@@ -610,7 +610,7 @@
  Concepts assume different roles in a data structure definition:
--* //dimensions// are required to uniquely identify an observation (a data value); e.g., for time series, at least one geographic, one temporal, and one (“mixed") subject-matter dimension are required to identify a data value (for instance: reference area = Mexico, time = 2002, indicator = GDP nominal, US$){{footnote}}Please note that this is not a recommendation to always have three dimensions only. This is just a simplified example.{{/footnote}};
++* //dimensions// are required to uniquely identify an observation (a data value); e.g., for time series, at least one geographic, one temporal, and one (“mixed") subject-matter dimension are required to identify a data value (for instance: reference area = Mexico, time = 2002, indicator = GDP nominal, US$)[[(% class="wikiinternallink wikiinternallink wikiinternallink" %)^^~[6~]^^>>path:#_ftn6]](%%);
  * //measures// are the containers of the actual observation or data values;
  * //attributes// provide additional meta-information required to interpret the data correctly but not to identify the observations; for instance, data for the same observation defined by a value combination of the dimensions (also termed “key”) will usually only be provided for one unit multiplier, e.g. in millions; hence unit multiplier is not necessary to identify an observation, but it is still required for a proper interpretation. Attributes can be defined as mandatory or not mandatory, and they can be attached at different levels, e.g. at observation level or at the level of groups defined by the value combinations of a predefined subset of dimensions (for example reporting currency may be attached at the country level).
@@ -650,6 +650,19 @@
  METIS: Generic Statistical Business Process Model (GSBPM). Available online at http:~/~/www1.unece.org/stat/platform/display/metis/The+Generic+Statistical+Business+Process+Model. UN's System of National Accounts Manual 2008 (SNA2008). Available online at http:~/~/unstats.un.org/unsd/nationalaccount/sna2008.asp.
++
  ----
++[[~[2~]>>path:#_ftnref2]] A Cartesian product (or product set) is a mathematical construct that builds a new set out of a number of given sets. Each member of the Cartesian product corresponds to the selection of one element each in every one of the original sets.
++
++[[~[3~]>>path:#_ftnref3]] In case a structure map is used to define reduced versions of the DSD, the number of unmapped dimensions is the equivalent measure of sparseness.
++
++[[~[4~]>>path:#_ftnref4]] Please note that the example is taken from the development status of the BOP DSD at the time of writing this document. The concepts and their relevance for certain data exchanges (represented as data flows or derived DSDs) may be different in the final version of the DSD.
++
++[[~[5~]>>path:#_ftnref5]] For other more technical requirements such as the admissible characters in a code or label see the SDMX technical documents.
++
++[[~[6~]>>path:#_ftnref6]] Please note that this is not a recommendation to always have three dimensions only. This is just a simplified example.
++
++----
++
  {{putFootnotes/}}

1768468989793-901.png

Author

...	...	@@ -1,1 +1,0 @@
1		-xwiki:XWiki.helena

Size

...	...	@@ -1,1 +1,0 @@
1		-52.9 KB

Content

1768469016538-287.png

Author

...	...	@@ -1,1 +1,0 @@
1		-xwiki:XWiki.helena

Size

...	...	@@ -1,1 +1,0 @@
1		-42.6 KB

Content