Skip to Content
Last modified by Elena on 2025/09/10 11:19
From version 2.20
edited by Helena
on 2025/07/02 15:26
Change comment: There is no comment for this version
To version 2.26
edited by Helena
on 2025/07/02 15:31
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -873,53 +873,54 @@
873 873  
874 874  The following convention is used to indicate the state of components in data messages:
875 875  
876 -(% style="width:1591.29px" %)
877 -| | |**Dimension value is**|(% style="width:384px" %) |(% style="width:598px" %)**Measure, attribute or reference metadata value is**|(% style="width:201px" %)
878 -| | |**Omitted**|(% style="width:384px" %)**switched off**|(% style="width:598px" %)**Omitted**|(% style="width:201px" %)**Present**
879 -|Action|Delete|bulk deletion: dimension value doesn't matter|(% style="width:384px" %)only for irrelevant dimensions:1) higher-level (reference metadata) attributes not attached to this dimension(incl. TIME\_PERIOD)2) measures and attributes not attached to this dimension if the DSD allows for an ‘evolving structure’ (excl. TIME\_PERIOD)|(% style="width:598px" %)to be deleted only if **all** non-dimension components are omitted|(% style="width:201px" %)to be deleted
880 -| |Merge|//bulk merge is not permitted//|(% style="width:384px" %)(see above)|(% style="width:598px" %)not to be changed|(% style="width:201px" %)to be updated/inserted
881 -| |Replace|//bulk replace is not permitted//|(% style="width:384px" %)(see above)|(% style="width:598px" %)at permitted replacement levels: to be deleted, otherwise not to be changed|(% style="width:201px" %)to be updated/inserted
882 -|Format|XML|xml element/attribute is absent|(% style="width:384px" %)~|xml element/attribute is absent|(% style="width:598px" %)any valid or intentionally missing value|(% style="width:201px" %)
883 -| |JSON|\<empty\>|(% style="width:384px" %)““|(% style="width:598px" %)NULL or absent|(% style="width:201px" %)(see above)
884 -| |CSV|\<empty\> cell or column is absent|(% style="width:384px" %)~|<empty> cell or column is absent|(% style="width:598px" %)(see above)|(% style="width:201px" %)
876 +(% style="width:1364.29px" %)
877 +| | |(% style="width:260px" %)**Dimension value is**|(% style="width:391px" %) |(% style="width:383px" %)**Measure, attribute or reference metadata value is**|(% style="width:184px" %)
878 +| | |(% style="width:260px" %)**Omitted**|(% style="width:391px" %)**switched off**|(% style="width:383px" %)**Omitted**|(% style="width:184px" %)**Present**
879 +|Action|Delete|(% style="width:260px" %)bulk deletion: dimension value doesn't matter|(% style="width:391px" %)only for irrelevant dimensions:1) higher-level (reference metadata) attributes not attached to this dimension(incl. TIME\_PERIOD)2) measures and attributes not attached to this dimension if the DSD allows for an ‘evolving structure’ (excl. TIME\_PERIOD)|(% style="width:383px" %)to be deleted only if **all** non-dimension components are omitted|(% style="width:184px" %)to be deleted
880 +| |Merge|(% style="width:260px" %)//bulk merge is not permitted//|(% style="width:391px" %)(see above)|(% style="width:383px" %)not to be changed|(% style="width:184px" %)to be updated/inserted
881 +| |Replace|(% style="width:260px" %)//bulk replace is not permitted//|(% style="width:391px" %)(see above)|(% style="width:383px" %)at permitted replacement levels: to be deleted, otherwise not to be changed|(% style="width:184px" %)to be updated/inserted
882 +|Format|XML|(% style="width:260px" %)xml element/attribute is absent|(% style="width:391px" %)~|xml element/attribute is absent|(% style="width:383px" %)any valid or intentionally missing value|(% style="width:184px" %)
883 +| |JSON|(% style="width:260px" %)\<empty\>|(% style="width:391px" %)““|(% style="width:383px" %)NULL or absent|(% style="width:184px" %)(see above)
884 +| |CSV|(% style="width:260px" %)\<empty\> cell or column is absent|(% style="width:391px" %)~|<empty> cell or column is absent|(% style="width:383px" %)(see above)|(% style="width:184px" %)
885 885  
886 886  **Important notes:**
887 887  
888 -The terms “*delete*”, “*merge*” and “*replace*” do **not** imply a physical replacement or deletion of values in the underlying database. To minimize the physical resource requirements, SDMX web service implementations that do not support the *includeHistory* and *asOf* URL parameters might physically replace the existing values in the database. SDMX web services that neither support the *updatedAfter* URL parameter might also implement physical deletions. However, SDMX web services that support these parameters (or other time-machine features), would not overwrite or delete the physical values.
888 +The terms “//delete//”, “//merge//” and “//replace//” do **not** imply a physical replacement or deletion of values in the underlying database. To minimize the physical resource requirements, SDMX web service implementations that do not support the //includeHistory// and //asOf// URL parameters might physically replace the existing values in the database. SDMX web services that neither support the *updatedAfter* URL parameter might also implement physical deletions. However, SDMX web services that support these parameters (or other time-machine features), would not overwrite or delete the physical values.
889 889  
890 -SDMX web services that support the *includeHistory* or *asOf* URL parameters should never allow deleting their **historic** data content because this would interfere with the interests of data consumers, such as data aggregators. Therefore, a specific feature to physically delete previous (outdated) content is intentionally not added to the SDMX standard syntax. If such a feature is required by an organisation, then it needs to be implemented as a custom feature outside the SDMX standard.
890 +SDMX web services that support the //includeHistory// or //asOf// URL parameters should never allow deleting their **historic** data content because this would interfere with the interests of data consumers, such as data aggregators. Therefore, a specific feature to physically delete previous (outdated) content is intentionally not added to the SDMX standard syntax. If such a feature is required by an organisation, then it needs to be implemented as a custom feature outside the SDMX standard.
891 891  
892 -Likewise, all SDMX-compliant systems that do (or are configured to) support the *updatedAfter* URL parameter need to systematically retain the information about deleted data (or data-related reference metadata).
892 +Likewise, all SDMX-compliant systems that do (or are configured to) support the //updatedAfter// URL parameter need to systematically retain the information about deleted data (or data-related reference metadata).
893 893  
894 -All datasets – even with varying actions – within a single data message have always to be treated as **ACID transaction** to guarantee “transactional safety” (full data consistency and validity despite errors, power failures, and other mishaps). These datasets are to be processed in the order of appearance in the message. The advantage of such data messages is thus the ability to bundle separate *delete* and *replace* or *merge* actions into one transactional data message.
894 +All datasets – even with varying actions – within a single data message have always to be treated as **ACID transaction** to guarantee “transactional safety” (full data consistency and validity despite errors, power failures, and other mishaps). These datasets are to be processed in the order of appearance in the message. The advantage of such data messages is thus the ability to bundle separate //delete// and //replace// or //merge// actions into one transactional data message.
895 895  
896 -**Recommended[^2] dataset actions in SDMX web service responses to GET data queries:**
896 +**Recommended{{footnote}}So far this is recommended for systems that do not require backward-compatibility. Later, with SDMX 4.0, this may generally be made mandatory{{/footnote}} dataset actions in SDMX web service responses to GET data queries:**
897 897  
898 -1. Without the *updatedAfter*, *includeHistory*, *detail*, *attributes* or *measures* URL parameters:
899 -
900 - The response message should contain the retrieved data in a *Replace* dataset (instead of the previous *information* dataset).
898 +~1. Without the //updatedAfter//, //includeHistory//, //detail//, //attributes// or //measures// URL parameters:
901 901  
902 -1. Without the *updatedAfter* and *includeHistory*, but with *detail*, *attributes* or *measures* URL parameters:
903 -
904 - The response message should contain the retrieved data in a *Merge* dataset (instead of the previous *Information* dataset).
900 +The response message should contain the retrieved data in a *Replace* dataset (instead of the previous *information* dataset).
905 905  
906 -1. With the *updatedAfter* URL parameter:
907 -
908 - The response must include the information of all previously updated, inserted and deleted data or data-related reference metadata, even if bulk deletions have been used. One of the two approaches are possible:
902 +2. Without the *updatedAfter* and *includeHistory*, but with *detail*, *attributes* or *measures* URL parameters:
909 909  
910 -* a *Delete* dataset for entirely deleted observations and for entirely deleted sets of (data-related reference metadata) attribute values attached to specific dimension combinations and
911 - a *Replace* dataset for all other changed observations and changed attribute and data-related reference metadata values attached to specific dimension combinations, or 
912 -* a *Delete* dataset for entirely deleted observations, for entirely deleted sets of (data-related reference metadata) attribute values attached to specific dimension combinations and for individually deleted mesure, attribute and reference metadata values and
913 - a *Merge* dataset for all other updated or inserted observation, attribute and data-related reference metadata values.
914 -
904 +The response message should contain the retrieved data in a *Merge* dataset (instead of the previous *Information* dataset).
905 +
906 +3. With the *updatedAfter* URL parameter:
907 +
908 +The response must include the information of all previously updated, inserted and deleted data or data-related reference metadata, even if bulk deletions have been used. One of the two approaches are possible:
909 +
910 +* a //Delete// dataset for entirely deleted observations and for entirely deleted sets of (data-related reference metadata) attribute values attached to specific dimension combinations and
911 +a //Replace// dataset for all other changed observations and changed attribute and data-related reference metadata values attached to specific dimension combinations, or 
912 +* a //Delete// dataset for entirely deleted observations, for entirely deleted sets of (data-related reference metadata) attribute values attached to specific dimension combinations and for individually deleted mesure, attribute and reference metadata values and
913 +a //Merge// dataset for all other updated or inserted observation, attribute and data-related reference metadata values.
914 +
915 +
915 915   The DB synchronization use case requires that the generated response must always allow achieving to replicate the exact same punctual data content as currently stored in the queried data source.
916 916  
917 917  1. With the *includeHistory* URL parameter:
918 -
919 +
919 919   Using a number of datasets with *Delete*, *Replace* or *Merge* actions and limited in their validity time span that allow achieving to replicate the exact same punctual data contents as previously stored in the queried data source.
920 920  
921 921  1. With the *asOf* URL parameter:
922 -
923 +
923 923   The recommendations of 1 and 2 apply depending on the other parameters. In addition, the returned dataset should have its validity time span limited to the point in time requested in the *asOf* parameter.
924 924  
925 925  [^2]: So far this is recommended for systems that do not require backward-compatibility. Later, with SDMX 4.0, this may generally be made mandatory.
... ... @@ -929,3 +929,5 @@
929 929  Reference metadata defined by a Metadataflow or a MetadataProvisionAgreement are exchanged within reference metadatasets, which are maintainable and thus for actions behave like structural metadata (artefacts): When interacting with SDMX Rest web services, the HTTP action verbs GET, PUT and POST are used to indicate the intended action per web request. Consequently, different actions cannot be bundled and executed with “transactional ACIDity”. Note that metadatasets retrieved using the HTTP header “Accept-Language” may contain only partial languages, and thus should be marked with its *isPartialLanguage* property set to true. Submitting such a partial metadataset to update an SDMX storage system will only add or update the included languages but not change other languages.
930 930  
931 931  The former message header or metadataset property *DataSetAction* is deprecated. To avoid conflicts, it is now ignored if still present.
933 +
934 +{{putFootnotes/}}