Changes for page Part IV. Data and Reference Metadata Namespaces

Last modified by Elena on 2025/09/10 11:19

From 2.14 to 2.15 From 2.23 to 2.24

From version 2.15

edited by Helena
on 2025/07/02 15:21

Change comment: There is no comment for this version

To version 2.23

edited by Helena
on 2025/07/02 15:28

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -801,16 +801,18 @@
  A complex type name ObsType must be created. Its content model will be derived via restriction of the base type dsd:ObsType. The complex type content model will be as follows:
--1. A sequence consisting of:
--1. An element reference to common:Annotations, with a minimum occurrence of 0
--   2. If any measures with complex representations are defined in the data structure, any attributes with complex representation that declare an attribute relationship with the observation, a Comp element with a form of unqualified, a minimum occurrence of 0, a maximum occurrence of unbounded, and a type of dsd:CompType
--   3. If any metadata attribute usages defined in the data structure that declares an attribute relationship with the observation, a local element named Metadata with the type metadata:MetadataSetType a form of unqualified, and a minimum occurrence of 0
++~1. A sequence consisting of:
++
++* i. An element reference to common:Annotations, with a minimum occurrence of 0
++* ii. If any measures with complex representations are defined in the data structure, any attributes with complex representation that declare an attribute relationship with the observation, a Comp element with a form of unqualified, a minimum occurrence of 0, a maximum occurrence of unbounded, and a type of dsd:CompType
++* iii. If any metadata attribute usages defined in the data structure that declares an attribute relationship with the observation, a local element named Metadata with the type metadata:MetadataSetType a form of unqualified, and a minimum occurrence of 0
++
 . An attribute named TIME\_PERIOD with a type of common:
     ObservationalTimePeriodValueType. If the dimension at the observation level is the time dimension (TIME\_PERIOD) or all dimensions and the time dimension is defined by the data structure, a usage of required; otherwise, a usage of prohibited
--3. If the dimension at the observation level is not all dimensions or the time dimension (TIME\_PERIOD), an attribute for the dimension at the observation level. The XML attribute [name](#331-Component-Name-Determination) and [type](#333-Simple--Primitive-Type-Determination) are defined according to the general rules defined in the previous section, and the usage is required
--4. If the dimension at the observation level is all dimensions, an attribute for each dimension defined by the data structure definition, except for the time dimension (TIME\_PERIOD). The XML attribute [name](#331-Component-Name-Determination) and [type](#333-Simple--Primitive-Type-Determination) are defined according to the general rules defined in the previous section, and the usage is required
--5. An attribute for each measure with simple representation defined by the data structure definition. The XML attribute [name](#331-Component-Name-Determination) and [type](#333-Simple--Primitive-Type-Determination) is defined according to the general rules defined in the previous section, and the usage is optional
--6. An attribute for each data attribute with simple representation defined in the data structure that declares an attribute relationship with the observation. The XML attribute [name](#331-Component-Name-Determination) and [type](#333-Simple--Primitive-Type-Determination) are defined according to the general rules defined in the previous section, and the usage is optional
++3. If the dimension at the observation level is not all dimensions or the time dimension (TIME\_PERIOD), an attribute for the dimension at the observation level. The XML attribute name and type are defined according to the general rules defined in the previous section, and the usage is required
++4. If the dimension at the observation level is all dimensions, an attribute for each dimension defined by the data structure definition, except for the time dimension (TIME\_PERIOD). The XML attribute name and type are defined according to the general rules defined in the previous section, and the usage is required
++5. An attribute for each measure with simple representation defined by the data structure definition. The XML attribute name and type is defined according to the general rules defined in the previous section, and the usage is optional
++6. An attribute for each data attribute with simple representation defined in the data structure that declares an attribute relationship with the observation. The XML attribute name and type are defined according to the general rules defined in the previous section, and the usage is optional
  === 3.4.6 CompType ===
@@ -841,7 +841,7 @@
  === 4.1.2 Replace Action ===
--Data or data-related reference metadata is to be replaced, through either update, insert or delete depending on already existing information. A full replacement is hereby assumed to take place at specific “replacement levels”: for entire observations and for any specific dimension combination for data-related reference metadata attributes. Within these “replacement levels” the provided values are inserted or updated, and omitted values are deleted. Values provided for the other attributes (those above the observation level) are merged (see *Merge* action).
++Data or data-related reference metadata is to be replaced, through either update, insert or delete depending on already existing information. A full replacement is hereby assumed to take place at specific “//replacement levels//”: for entire observations and for any specific dimension combination for data-related reference metadata attributes. Within these “replacement levels” the provided values are inserted or updated, and omitted values are deleted. Values provided for the other attributes (those above the observation level) are merged (see //Merge// action).
  Only non-dimensional components (measure, attribute or reference metadata values) can be **omitted** (null or absent). Bulk replacing is thus not supported.
@@ -849,9 +849,9 @@
  Replacing non-existing elements is not resulting in an error.
--All observations as well as the sets of data-related reference metadata attributes at specific dimension combinations impacted by the *Replace* action change their time stamp when used to update an SDMX storage system.
++All observations as well as the sets of data-related reference metadata attributes at specific dimension combinations impacted by the //Replace// action change their time stamp when used to update an SDMX storage system.
--Because the *replace* action always takes place at specific levels, it cannot be used to replace a whole dataset or a whole series. However, a “*replace all*” effect can be achieved by combining a *Delete* dataset containing a completely wildcarded key (where all dimension values are omitted) with a *Merge* or *Replace* dataset within the same data message. Similarly, to replace a whole series, a message can combine a *delete* dataset containing only the partial key of the series (where the not used dimension values are omitted) with a *Merge* or *Replace* dataset for that series.
++Because the //replace// action always takes place at specific levels, it cannot be used to replace a whole dataset or a whole series. However, a //“replace all”// effect can be achieved by combining a //Delete// dataset containing a completely wildcarded key (where all dimension values are omitted) with a //Merge// or //Replace// dataset within the same data message. Similarly, to replace a whole series, a message can combine a //delete// dataset containing only the partial key of the series (where the not used dimension values are omitted) with a //Merge// or //Replace// dataset for that series.
  === 4.1.3 Delete Action ===
@@ -871,24 +871,25 @@
  The following convention is used to indicate the state of components in data messages:
--| | |**Dimension value is**| |**Measure, attribute or reference metadata value is**| |
--| | |**Omitted**|**switched off**|**Omitted**|**Present**|
--|Action|Delete|bulk deletion: dimension value doesn't matter|only for irrelevant dimensions:1) higher-level (reference metadata) attributes not attached to this dimension(incl. TIME\_PERIOD)2) measures and attributes not attached to this dimension if the DSD allows for an ‘evolving structure’ (excl. TIME\_PERIOD)|to be deleted only if **all** non-dimension components are omitted|to be deleted|
--| |Merge|*bulk merge is not permitted*|(see above)|not to be changed|to be updated/inserted|
--| |Replace|*bulk replace is not permitted*|(see above)|at permitted replacement levels: to be deleted, otherwise not to be changed|to be updated/inserted|
--|Format|XML|xml element/attribute is absent|~|xml element/attribute is absent|any valid or intentionally missing value|
--| |JSON|\<empty\>|““|NULL or absent|(see above)|
--| |CSV|\<empty\> cell or column is absent|~|<empty> cell or column is absent|(see above)|
++(% style="width:1364.29px" %)
++| | |(% style="width:260px" %)**Dimension value is**|(% style="width:391px" %) |(% style="width:383px" %)**Measure, attribute or reference metadata value is**|(% style="width:184px" %)
++| | |(% style="width:260px" %)**Omitted**|(% style="width:391px" %)**switched off**|(% style="width:383px" %)**Omitted**|(% style="width:184px" %)**Present**
++|Action|Delete|(% style="width:260px" %)bulk deletion: dimension value doesn't matter|(% style="width:391px" %)only for irrelevant dimensions:1) higher-level (reference metadata) attributes not attached to this dimension(incl. TIME\_PERIOD)2) measures and attributes not attached to this dimension if the DSD allows for an ‘evolving structure’ (excl. TIME\_PERIOD)|(% style="width:383px" %)to be deleted only if **all** non-dimension components are omitted|(% style="width:184px" %)to be deleted
++| |Merge|(% style="width:260px" %)//bulk merge is not permitted//|(% style="width:391px" %)(see above)|(% style="width:383px" %)not to be changed|(% style="width:184px" %)to be updated/inserted
++| |Replace|(% style="width:260px" %)//bulk replace is not permitted//|(% style="width:391px" %)(see above)|(% style="width:383px" %)at permitted replacement levels: to be deleted, otherwise not to be changed|(% style="width:184px" %)to be updated/inserted
++|Format|XML|(% style="width:260px" %)xml element/attribute is absent|(% style="width:391px" %)~|xml element/attribute is absent|(% style="width:383px" %)any valid or intentionally missing value|(% style="width:184px" %)
++| |JSON|(% style="width:260px" %)\<empty\>|(% style="width:391px" %)““|(% style="width:383px" %)NULL or absent|(% style="width:184px" %)(see above)
++| |CSV|(% style="width:260px" %)\<empty\> cell or column is absent|(% style="width:391px" %)~|<empty> cell or column is absent|(% style="width:383px" %)(see above)|(% style="width:184px" %)
  **Important notes:**
--The terms “*delete*”, “*merge*” and “*replace*” do **not** imply a physical replacement or deletion of values in the underlying database. To minimize the physical resource requirements, SDMX web service implementations that do not support the *includeHistory* and *asOf* URL parameters might physically replace the existing values in the database. SDMX web services that neither support the *updatedAfter* URL parameter might also implement physical deletions. However, SDMX web services that support these parameters (or other time-machine features), would not overwrite or delete the physical values.
++The terms “//delete//”, “//merge//” and “//replace//” do **not** imply a physical replacement or deletion of values in the underlying database. To minimize the physical resource requirements, SDMX web service implementations that do not support the //includeHistory// and //asOf// URL parameters might physically replace the existing values in the database. SDMX web services that neither support the *updatedAfter* URL parameter might also implement physical deletions. However, SDMX web services that support these parameters (or other time-machine features), would not overwrite or delete the physical values.
--SDMX web services that support the *includeHistory* or *asOf* URL parameters should never allow deleting their **historic** data content because this would interfere with the interests of data consumers, such as data aggregators. Therefore, a specific feature to physically delete previous (outdated) content is intentionally not added to the SDMX standard syntax. If such a feature is required by an organisation, then it needs to be implemented as a custom feature outside the SDMX standard.
++SDMX web services that support the //includeHistory// or //asOf// URL parameters should never allow deleting their **historic** data content because this would interfere with the interests of data consumers, such as data aggregators. Therefore, a specific feature to physically delete previous (outdated) content is intentionally not added to the SDMX standard syntax. If such a feature is required by an organisation, then it needs to be implemented as a custom feature outside the SDMX standard.
--Likewise, all SDMX-compliant systems that do (or are configured to) support the *updatedAfter* URL parameter need to systematically retain the information about deleted data (or data-related reference metadata).
++Likewise, all SDMX-compliant systems that do (or are configured to) support the //updatedAfter// URL parameter need to systematically retain the information about deleted data (or data-related reference metadata).
--All datasets – even with varying actions – within a single data message have always to be treated as **ACID transaction** to guarantee “transactional safety” (full data consistency and validity despite errors, power failures, and other mishaps). These datasets are to be processed in the order of appearance in the message. The advantage of such data messages is thus the ability to bundle separate *delete* and *replace* or *merge* actions into one transactional data message.
++All datasets – even with varying actions – within a single data message have always to be treated as **ACID transaction** to guarantee “transactional safety” (full data consistency and validity despite errors, power failures, and other mishaps). These datasets are to be processed in the order of appearance in the message. The advantage of such data messages is thus the ability to bundle separate //delete// and //replace// or //merge// actions into one transactional data message.
  **Recommended[^2] dataset actions in SDMX web service responses to GET data queries:**