Changes for page Guidelines for Confidentiality and Embargo in SDMX

Last modified by Artur K. on 2026/05/29 14:28

From 2.2 to 2.1

From version 2.1

edited by Helena K.
on 2026/01/27 12:56

Change comment: There is no comment for this version

To version 1.1

edited by Helena K.
on 2026/01/27 12:51

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -19,18 +19,25 @@
  This paper presents use case scenarios related to confidentiality and embargo in SDMX data exchanges, and provides recommendations on how to represent these elements in the SDMX model.  The aim is to provide a consistent and practical way to represent these aspects in SDMX artefacts in order to promote cross-domain consistency, and harmonise methodology and processes.
++
  Confidentiality aims at protecting data from unauthorised disclosure that could be prejudicial or harmful to the interest of the source or other relevant parties.
++
  Embargo means that data may become public only after expiry of a pre-defined date and time.
++
  Embargo establishes a relationship between a set of data (e.g. an observation), a date/time and a group of privileged data recipients.
++
  Disclosure of data marked as confidential or under embargo is not permitted. Procedures should be in place to prevent such disclosure, including rules for staff, aggregation rules when disseminating data, provision of unit records, etc.
++
  There needs to be a formal agreement between organisations involved in the exchange of confidential data in order to prepare systems and workflows.
++
  Data exchange partners are advised to agree up front on the usage of the embargo mechanism(s) for specific data messages.
++
  The embargo CONF_STATUS value “E” is not recommended for final dissemination to users but only for data exchange.
  = Use Cases =
@@ -41,6 +41,7 @@
  Data is available to the public immediately, meaning that data is not confidential and there is no embargo.
++
  The data’s CONF_STATUS attribute should be set to “Free (free for publication)”.
  |(((
@@ -55,24 +55,28 @@
  One or more observations in the data message are confidential.  Embargo does not play a role in this scenario.  Depending on arrangements between data exchange partners, this data can be made available to privileged data users.
--The observation’s CONF_STATUS attribute should use a specific code denoting the confidential character of the information.  Below are some examples of such confidentiality statuses{{footnote}}For a full list of confidentiality statuses, see https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx{{/footnote}}:
++The observation’s CONF_STATUS attribute should use a specific code denoting the confidential character of the information.  Below are some examples of such confidentiality statuses[[~[1~]>>path:#_ftn1]]:
++
  * **N**:   Not for publication, restricted for internal use only.  Used to denote observations that are restricted for internal use only within organisations
  * **C**:   Confidential statistical information (primary confidentiality) due to identifiable respondents
  * **D**:   Secondary confidentiality set by the sender, not for publication
  * **A**:   Primary confidentiality due to small counts
++
++
  === Forwarding confidential data to secondary recipients ===
  A sender sends confidential data to certain primary recipients, and allows those to forward the confidential data to a restricted and pre-defined set of secondary recipients.
--The observation’s CONF_STATUS attribute should be marked as “Not for publication, restricted for internal use only”.  An additional observation-level attribute: CONF_REDIST, defines the secondary recipient(s) to whom the sender allows the primary recipient to forward confidential data{{footnote}}Example: National statistical institute XX reporting data to Eurostat indicates that Eurostat can forward those data to the ECB, IMF and OECD.  More complex use case: The reporting organization specifies that Eurostat can forward those data only to the ECB Statistics Department, thus excluding all other organisations as well as all other ECB departments.{{/footnote}}.  See section **Use of the CONF_REDIST attribute** for the appropriate coding of this attribute.
++The observation’s CONF_STATUS attribute should be marked as “Not for publication, restricted for internal use only”.  An additional observation-level attribute: CONF_REDIST, defines the secondary recipient(s) to whom the sender allows the primary recipient to forward confidential data[[~[2~]>>path:#_ftn2]].  See section **Use of the CONF_REDIST attribute** for the appropriate coding of this attribute.
++
++
  The forwarding of confidential data is represented as follows in SDMX:
  |(((
--(% class="wikigeneratedid" id="HSDMXrepresentation-1" %)
--SDMX representation
++=== SDMX representation ===
  * **CONF_STATUS**: N;
  * **CONF_REDIST **(Observation, Conditional): [Organisation(s)];
@@ -87,13 +87,14 @@
  * Allowing privileged access to embargoed data
  * Enabling the frontloading of data into systems
++
++
  **//Allowing privileged access to embargoed data//**
  If the goal is to allow the data recipient to have privileged access to embargoed observations in a data message (message), the embargoed observation’s CONF_STATUS attribute should be coded as “E: Not for publication until the embargo time expires; free for publication after the embargo time expires.” with an observation level attribute EMBARGO_TIME (date/time/time zone).
  |(((
--(% class="wikigeneratedid" id="HSDMXrepresentation-2" %)
--SDMX representation
++=== SDMX representation ===
  * **CONF_STATUS**: E;
  * **EMBARGO**_**TIME** (Observation, Conditional): [timestamp]
@@ -106,21 +106,26 @@
  * (Recommended) With UTC indicator: 2017-12-15T14:02:29Z
  * With timezone indicator: 2017-12-15T15:02:29+01:00
++
++
  **//Enabling the frontloading of data into systems//**
  If the goal is to allow frontloading of a whole data message into systems so that the data can be made visible to users at the expiry of the embargo date/time, the header section of the message should contain an embargo date/time attribute.  This implies that all information in the data message is under the embargo date/time set in the header.  The header attribute EmbargoDate with format date/time/time zone indicates until when the whole data message received cannot be shared with any recipient users.
++
  Once the EmbargoDate in the header elapses, each observation’s confidentiality status becomes that which is marked in the CONF_STATUS attributes.
++
  Note that this scenario presumes that all data in the message cannot be viewed before the header EmbargoDate, and that there is no privileged access before this time.  However, observations may be marked with any other confidentiality status that is valid after the frontloading EmbargoDate elapses.
  |(((
--(% class="wikigeneratedid" id="HSDMXRepresentation" %)
--SDMX Representation
++=== SDMX Representation ===
  * **CONF_STATUS**: <Set to the required confidentiality status after the embargo time elapses>; <Header>\<EmbargoDate>: [timestamp]
  )))
++
++
  The two ways of representing embargoed data exist to provide efficiency in the exchange, allow for differentiating data intended to be frontloaded and data aimed to be provided in advance to a restricted audience, and provide flexibility when few observations need to be embargoed in a large data message.  The trade-off is the complication of system implementation to support the two representations of embargo, which has to be done locally on a case-by-case basis.
  = Additional recommendations and examples =
@@ -129,12 +129,18 @@
  === Use of the CONF_REDIST attribute ===
++
  The CONF_REDIST attribute defines the secondary recipient(s) to whom the sender allows the primary recipient to forward confidential data.  It is recommended to be an optional attribute at observation level.  Ideally it should reference a shared code list containing standard organisation codes.  To allow several secondary recipients there are these possibilities:
--Use a code that represents multiple organisations, or;
--Use several CONF_REDIST attributes to portray the multiple recipients.  Each attribute represents one recipient and references the same codelist.  This implementation is cleaner than the above point 1, though this will require adding as many attributes to your DSD as there are potential recipients of the redistributed confidential data.
++1. Use a code that represents multiple organisations, or;
++
++
++1. Use several CONF_REDIST attributes to portray the multiple recipients.  Each attribute represents one recipient and references the same codelist.  This implementation is cleaner than the above point 1, though this will require adding as many attributes to your DSD as there are potential recipients of the redistributed confidential data.
++
++
++
  If the EMBARGO_TIME and CONF_REDIST attributes are both used:
 . Data is available only to the organisations in CONF_REDIST until EMBARGO_TIME
@@ -141,8 +141,7 @@
 . Data is available to the public after EMBARGO_TIME
  |(% colspan="3" %)(((
--(% class="wikigeneratedid" id="HPrivilegedAccess" %)
--Privileged Access
++= Privileged Access =
  )))
  |**Use case**|**No forwarding**|**Forwarding**
  |**Embargo**|(((
@@ -171,6 +171,8 @@
  * The national statistical institutes send data to Eurostat, and allow the data to be shared with the ECB for statistical coproduction
  * The data may only be shared with the public on the next day
++
++
  **CONF_STATUS:**E**;**
  **CONF_REDIST: **ECB**;**
@@ -177,31 +177,27 @@
  **EMBARGO_TIME=<**T+1 day**, **e.g.** **2017-12-15T10:00:00Z>
++
  The solutions suggested above aim at covering the most common confidentiality and embargo use cases within a single transmission from the primary reporter to the primary recipient.  However, for some more complex scenarios it might still be required to make multiple transmissions.
++
  It is strongly recommended that use cases are specified in an agreement between organisations involved in regular transmissions up-front in order to avoid unnecessary delay in data publication or – much worse – confidentiality breaches.
  **Annex 1: SDMX Representation of the confidentiality use cases**
  |(((
--(% class="wikigeneratedid" id="HUsecase" %)
--Use case
++== Use case ==
  )))|(((
--(% class="wikigeneratedid" id="HCONF_STATUS28Observation29" %)
--CONF_STATUS (Observation)
++== CONF_STATUS (Observation) ==
  )))|(((
--(% class="wikigeneratedid" id="HAdditionalattributes" %)
--Additional attributes
++== Additional attributes ==
  )))|(((
--(% class="wikigeneratedid" id="HRemarks" %)
--Remarks
++== Remarks ==
  )))
  |(((
--(% class="wikigeneratedid" id="HNon-confidentialdata" %)
--Non-confidential data
++== Non-confidential data ==
  )))|(((
--(% class="wikigeneratedid" id="HF" %)
--F
++== F ==
  )))|(((
  ==   ==
  )))|(((
@@ -208,72 +208,58 @@
  ==   ==
  )))
  |(((
--(% class="wikigeneratedid" id="HConfidentialdatawithnoembargo" %)
--Confidential data with no embargo
++== Confidential data with no embargo ==
  )))|(((
--(% class="wikigeneratedid" id="HC3BD3BS3BA3BO3BT3BG3BM3BN" %)
--C;D;S;A;O;T;G;M;N
++== C;D;S;A;O;T;G;M;N ==
  )))|(((
--(% class="wikigeneratedid" id="H-2" %)
--
++==   ==
  )))|(((
--(% class="wikigeneratedid" id="HCONF_STATUSwillusuallybeCbutmayalsobeD3BS3BA3BO3BT3BG3BM3BNdependingontherequiredstatusandconfidentialityreason.A0SeetheCL_CONF_STATUScodelistfordetails5B35D" %)
--CONF_STATUS will usually be C but may also be D;S;A;O;T;G;M;N depending on the required status and confidentiality reason.  See the CL_CONF_STATUS code list for details{{footnote}}https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx{{/footnote}}
++== CONF_STATUS will usually be C but may also be D;S;A;O;T;G;M;N depending on the required status and confidentiality reason.  See the CL_CONF_STATUS code list for details[[~[3~]>>path:#_ftn3]] ==
  )))
--|(((
--**Forwarding of confidential data**
--)))|(((
--N
--)))|(((
++|**Forwarding of confidential data**|N|(((
  CONF_REDIST: (Observation, Conditional)
  )))|CONF_REDIST may represent multiple organisations
  |(((
--(% class="wikigeneratedid" id="HEmbargo:Privilegedaccess" %)
--Embargo: Privileged access
++== Embargo: Privileged access ==
  )))|(((
--(% class="wikigeneratedid" id="HE" %)
--E
++== E ==
  )))|(((
--(% class="wikigeneratedid" id="HEMBARGO_TIME28Observation2CConditional29" %)
--EMBARGO_TIME (Observation, Conditional)
++== EMBARGO_TIME (Observation, Conditional) ==
  )))|Only the observations with an EMBARGO_TIME attribute are embargoed. After the embargo time elapses, the data are free for publication (equivalent to F status).
  |(((
--(% class="wikigeneratedid" id="HEmbargo:Privilegedaccesswithforwarding" %)
--Embargo: Privileged access with forwarding
++== Embargo: Privileged access with forwarding ==
  )))|(((
--(% class="wikigeneratedid" id="HE-1" %)
--E
++== E ==
  )))|(((
  EMBARGO_TIME (Observation, Conditional)
--(% class="wikigeneratedid" id="HCONF_REDIST:28Observation2CConditional29" %)
--CONF_REDIST: (Observation, Conditional)
++== CONF_REDIST: (Observation, Conditional) ==
  )))|(((
--(% class="wikigeneratedid" id="HOnlytheobservationswithanEMBARGO_TIMEattributeareembargoed.Aftertheembargotimeelapses2Cthedataarefreeforpublication28equivalenttoFstatus29." %)
--Only the observations with an EMBARGO_TIME attribute are embargoed.  After the embargo time elapses, the data are free for publication (equivalent to F status).
++== Only the observations with an EMBARGO_TIME attribute are embargoed.  After the embargo time elapses, the data are free for publication (equivalent to F status). ==
--(% class="wikigeneratedid" id="HCONF_REDISTmayrepresentmultipleorganisations" %)
--CONF_REDIST may represent multiple organisations
++== CONF_REDIST may represent multiple organisations ==
  )))
  |(((
--(% class="wikigeneratedid" id="HEmbargo:Frontloading" %)
--Embargo: Frontloading
++== Embargo: Frontloading ==
  )))|(((
--(% class="wikigeneratedid" id="HSettotherequiredconfidentialitystatusaftertheembargotimeelapses." %)
--Set to the required confidentiality status after the embargo time elapses.
++== Set to the required confidentiality status after the embargo time elapses. ==
  )))|(((
--(% class="wikigeneratedid" id="H3CHeader5CEmbargoDate3E:5Btimestamp5D" %)
--<Header\EmbargoDate>: [timestamp]
++== <Header\EmbargoDate>: [timestamp] ==
--)))|(((
--There is no EMBARGO_TIME attribute as the whole message is embargoed with no privileged access.
--)))
++)))|There is no EMBARGO_TIME attribute as the whole message is embargoed with no privileged access.
++
++
++
  ----
-- {{putFootnotes/}}
++[[~[1~]>>path:#_ftnref1]] For a full list of confidentiality statuses, see [[https:~~/~~/sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx>>url:https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx]].
++
++[[~[2~]>>path:#_ftnref2]] Example: National statistical institute XX reporting data to Eurostat indicates that Eurostat can forward those data to the ECB, IMF and OECD.  More complex use case: The reporting organization specifies that Eurostat can forward those data only to the ECB Statistics Department, thus excluding all other organisations as well as all other ECB departments.
++
++[[~[3~]>>path:#_ftnref3]]  [[https:~~/~~/sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx>>url:https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx]]