Changes for page Guidelines for Confidentiality and Embargo in SDMX
Last modified by Artur K. on 2026/05/29 14:28
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -4,12 +4,12 @@ 4 4 5 5 = Document History = 6 6 7 -|Version|Date|Comment 8 -|1.0|21/8/2015|Initial version. 9 -|2.0|7/3/2018|((( 7 +(% style="width:1039.96px" %) 8 +|Version|(% style="width:110px" %)Date|(% style="width:856px" %)Comment 9 +|1.0|(% style="width:110px" %)21/8/2015|(% style="width:856px" %)Initial version. 10 +|2.0|(% style="width:110px" %)7/3/2018|(% style="width:856px" %)((( 10 10 Replaced the “Embargo: Privileged access” use case confidentiality status to use CONF_STATUS:E instead of CONF_STATUS:N. When this guideline is implemented, the CONF_STATUS:N can no longer be used for this use case (the embargo time is ignored if the CONF_STATUS is N). 11 11 12 - 13 13 Clarified the document text, removed superfluous text. 14 14 15 15 Added use of time zone is recommended. ... ... @@ -17,7 +17,7 @@ 17 17 18 18 = Introduction = 19 19 20 -This paper presents use case scenarios related to confidentiality and embargo in SDMX data exchanges, and provides recommendations on how to represent these elements in the SDMX model. The aim is to provide a consistent and practical way to represent these aspects in SDMX artefacts in order to promote cross-domain consistency, and harmonise methodology and processes.20 +This paper presents use case scenarios related to confidentiality and embargo in SDMX data exchanges, and provides recommendations on how to represent these elements in the SDMX model. The aim is to provide a consistent and practical way to represent these aspects in SDMX artefacts in order to promote cross-domain consistency, and harmonise methodology and processes. 21 21 22 22 Confidentiality aims at protecting data from unauthorised disclosure that could be prejudicial or harmful to the interest of the source or other relevant parties. 23 23 ... ... @@ -35,7 +35,7 @@ 35 35 36 36 = Use Cases = 37 37 38 -This section describes the confidentiality and embargo use cases that are addressed by these guidelines. The use cases and embargo SDMX representations are summarised in annex 1:38 +This section describes the confidentiality and embargo use cases that are addressed by these guidelines. The use cases and embargo SDMX representations are summarised in annex 1: 39 39 40 40 == Use case 1: Non-confidential data == 41 41 ... ... @@ -53,20 +53,20 @@ 53 53 54 54 === Exchange of confidential data without embargo nor forwarding to secondary recipients === 55 55 56 -One or more observations in the data message are confidential. Embargo does not play a role in this scenario.Depending on arrangements between data exchange partners, this data can be made available to privileged data users.56 +One or more observations in the data message are confidential. Embargo does not play a role in this scenario. Depending on arrangements between data exchange partners, this data can be made available to privileged data users. 57 57 58 -The observation’s CONF_STATUS attribute should use a specific code denoting the confidential character of the information. Below are some examples of such confidentiality statuses[[~[1~]>>path:#_ftn1]]:58 +The observation’s CONF_STATUS attribute should use a specific code denoting the confidential character of the information. Below are some examples of such confidentiality statuses{{footnote}}For a full list of confidentiality statuses, see https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx{{/footnote}}: 59 59 60 -* **N**: Not for publication, restricted for internal use only.Used to denote observations that are restricted for internal use only within organisations61 -* **C**: Confidential statistical information (primary confidentiality) due to identifiable respondents62 -* **D**: Secondary confidentiality set by the sender, not for publication63 -* **A**: Primary confidentiality due to small counts60 +* **N**: Not for publication, restricted for internal use only. Used to denote observations that are restricted for internal use only within organisations 61 +* **C**: Confidential statistical information (primary confidentiality) due to identifiable respondents 62 +* **D**: Secondary confidentiality set by the sender, not for publication 63 +* **A**: Primary confidentiality due to small counts 64 64 65 65 === Forwarding confidential data to secondary recipients === 66 66 67 67 A sender sends confidential data to certain primary recipients, and allows those to forward the confidential data to a restricted and pre-defined set of secondary recipients. 68 68 69 -The observation’s CONF_STATUS attribute should be marked as “Not for publication, restricted for internal use only”. An additional observation-level attribute: CONF_REDIST, defines the secondary recipient(s) to whom the sender allows the primary recipient to forward confidential data[[~[2~]>>path:#_ftn2]]. See section **Use of the CONF_REDIST attribute** for the appropriate coding of this attribute.69 +The observation’s CONF_STATUS attribute should be marked as “Not for publication, restricted for internal use only”. An additional observation-level attribute: CONF_REDIST, defines the secondary recipient(s) to whom the sender allows the primary recipient to forward confidential data{{footnote}}Example: National statistical institute XX reporting data to Eurostat indicates that Eurostat can forward those data to the ECB, IMF and OECD. More complex use case: The reporting organization specifies that Eurostat can forward those data only to the ECB Statistics Department, thus excluding all other organisations as well as all other ECB departments.{{/footnote}}. See section **Use of the CONF_REDIST attribute** for the appropriate coding of this attribute. 70 70 71 71 The forwarding of confidential data is represented as follows in SDMX: 72 72 ... ... @@ -99,7 +99,7 @@ 99 99 * **EMBARGO**_**TIME** (Observation, Conditional): [timestamp] 100 100 ))) 101 101 102 -Including a time zone is strongly recommended and the best case is to use the UTC (Coordinated Universal Time) time standard. However, if no time zone is provided then the time zone of the recipient is assumed.102 +Including a time zone is strongly recommended and the best case is to use the UTC (Coordinated Universal Time) time standard. However, if no time zone is provided then the time zone of the recipient is assumed. 103 103 104 104 These two examples represent the same time for a recipient established in the Central European time zone (e.g. Germany, Norway, Gibraltar): 105 105 ... ... @@ -106,46 +106,35 @@ 106 106 * (Recommended) With UTC indicator: 2017-12-15T14:02:29Z 107 107 * With timezone indicator: 2017-12-15T15:02:29+01:00 108 108 109 - 110 - 111 111 **//Enabling the frontloading of data into systems//** 112 112 113 -If the goal is to allow frontloading of a whole data message into systems so that the data can be made visible to users at the expiry of the embargo date/time, the header section of the message should contain an embargo date/time attribute. This implies that all information in the data message is under the embargo date/time set in the header.The header attribute EmbargoDate with format date/time/time zone indicates until when the whole data message received cannot be shared with any recipient users.111 +If the goal is to allow frontloading of a whole data message into systems so that the data can be made visible to users at the expiry of the embargo date/time, the header section of the message should contain an embargo date/time attribute. This implies that all information in the data message is under the embargo date/time set in the header. The header attribute EmbargoDate with format date/time/time zone indicates until when the whole data message received cannot be shared with any recipient users. 114 114 115 - 116 116 Once the EmbargoDate in the header elapses, each observation’s confidentiality status becomes that which is marked in the CONF_STATUS attributes. 117 117 115 +Note that this scenario presumes that all data in the message cannot be viewed before the header EmbargoDate, and that there is no privileged access before this time. However, observations may be marked with any other confidentiality status that is valid after the frontloading EmbargoDate elapses. 118 118 119 -Note that this scenario presumes that all data in the message cannot be viewed before the header EmbargoDate, and that there is no privileged access before this time. However, observations may be marked with any other confidentiality status that is valid after the frontloading EmbargoDate elapses. 120 - 121 121 |((( 122 -=== SDMX Representation === 118 +(% class="wikigeneratedid" id="HSDMXRepresentation" %) 119 +SDMX Representation 123 123 124 124 * **CONF_STATUS**: <Set to the required confidentiality status after the embargo time elapses>; <Header>\<EmbargoDate>: [timestamp] 125 125 ))) 126 126 124 +The two ways of representing embargoed data exist to provide efficiency in the exchange, allow for differentiating data intended to be frontloaded and data aimed to be provided in advance to a restricted audience, and provide flexibility when few observations need to be embargoed in a large data message. The trade-off is the complication of system implementation to support the two representations of embargo, which has to be done locally on a case-by-case basis. 127 127 128 - 129 -The two ways of representing embargoed data exist to provide efficiency in the exchange, allow for differentiating data intended to be frontloaded and data aimed to be provided in advance to a restricted audience, and provide flexibility when few observations need to be embargoed in a large data message. The trade-off is the complication of system implementation to support the two representations of embargo, which has to be done locally on a case-by-case basis. 130 - 131 131 = Additional recommendations and examples = 132 132 133 -In data flows that feature confidential data, CONF_STATUS is highly recommended to be a mandatory attribute. However, if CONF_STATUS is optional in the DSD and missing from an observation, it is always implied to be “F” (free).128 +In data flows that feature confidential data, CONF_STATUS is highly recommended to be a mandatory attribute. However, if CONF_STATUS is optional in the DSD and missing from an observation, it is always implied to be “F” (free). 134 134 135 135 === Use of the CONF_REDIST attribute === 136 136 132 +The CONF_REDIST attribute defines the secondary recipient(s) to whom the sender allows the primary recipient to forward confidential data. It is recommended to be an optional attribute at observation level. Ideally it should reference a shared code list containing standard organisation codes. To allow several secondary recipients there are these possibilities: 137 137 138 - TheCONF_REDISTattributedefines the secondary recipient(s)to whom the senderallowstheprimary recipient to forward confidential data. It isrecommendedtobe an optionalattribute at observationlevel.Ideally it shouldreference a shared code list containingstandard organisationcodes.Toallow several secondary recipients there are these possibilities:134 +Use a code that represents multiple organisations, or; 139 139 136 +Use several CONF_REDIST attributes to portray the multiple recipients. Each attribute represents one recipient and references the same codelist. This implementation is cleaner than the above point 1, though this will require adding as many attributes to your DSD as there are potential recipients of the redistributed confidential data. 140 140 141 -1. Use a code that represents multiple organisations, or; 142 - 143 - 144 - 145 -1. Use several CONF_REDIST attributes to portray the multiple recipients. Each attribute represents one recipient and references the same codelist. This implementation is cleaner than the above point 1, though this will require adding as many attributes to your DSD as there are potential recipients of the redistributed confidential data. 146 - 147 - 148 - 149 149 If the EMBARGO_TIME and CONF_REDIST attributes are both used: 150 150 151 151 1. Data is available only to the organisations in CONF_REDIST until EMBARGO_TIME ... ... @@ -152,7 +152,8 @@ 152 152 1. Data is available to the public after EMBARGO_TIME 153 153 154 154 |(% colspan="3" %)((( 155 -= Privileged Access = 144 +(% class="wikigeneratedid" id="HPrivilegedAccess" %) 145 +Privileged Access 156 156 ))) 157 157 |**Use case**|**No forwarding**|**Forwarding** 158 158 |**Embargo**|((( ... ... @@ -181,8 +181,6 @@ 181 181 * The national statistical institutes send data to Eurostat, and allow the data to be shared with the ECB for statistical coproduction 182 182 * The data may only be shared with the public on the next day 183 183 184 - 185 - 186 186 **CONF_STATUS:**E**;** 187 187 188 188 **CONF_REDIST: **ECB**;** ... ... @@ -189,27 +189,31 @@ 189 189 190 190 **EMBARGO_TIME=<**T+1 day**, **e.g.** **2017-12-15T10:00:00Z> 191 191 180 +The solutions suggested above aim at covering the most common confidentiality and embargo use cases within a single transmission from the primary reporter to the primary recipient. However, for some more complex scenarios it might still be required to make multiple transmissions. 192 192 193 -The solutions suggested above aim at covering the most common confidentiality and embargo use cases within a single transmission from the primary reporter to the primary recipient. However, for some more complex scenarios it might still be required to make multiple transmissions. 194 - 195 - 196 196 It is strongly recommended that use cases are specified in an agreement between organisations involved in regular transmissions up-front in order to avoid unnecessary delay in data publication or – much worse – confidentiality breaches. 197 197 198 198 **Annex 1: SDMX Representation of the confidentiality use cases** 199 199 200 200 |((( 201 -== Use case == 187 +(% class="wikigeneratedid" id="HUsecase" %) 188 +Use case 202 202 )))|((( 203 -== CONF_STATUS (Observation) == 190 +(% class="wikigeneratedid" id="HCONF_STATUS28Observation29" %) 191 +CONF_STATUS (Observation) 204 204 )))|((( 205 -== Additional attributes == 193 +(% class="wikigeneratedid" id="HAdditionalattributes" %) 194 +Additional attributes 206 206 )))|((( 207 -== Remarks == 196 +(% class="wikigeneratedid" id="HRemarks" %) 197 +Remarks 208 208 ))) 209 209 |((( 210 -== Non-confidential data == 200 +(% class="wikigeneratedid" id="HNon-confidentialdata" %) 201 +Non-confidential data 211 211 )))|((( 212 -== F == 203 +(% class="wikigeneratedid" id="HF" %) 204 +F 213 213 )))|((( 214 214 == == 215 215 )))|((( ... ... @@ -216,58 +216,72 @@ 216 216 == == 217 217 ))) 218 218 |((( 219 -== Confidential data with no embargo == 211 +(% class="wikigeneratedid" id="HConfidentialdatawithnoembargo" %) 212 +Confidential data with no embargo 220 220 )))|((( 221 -== C;D;S;A;O;T;G;M;N == 214 +(% class="wikigeneratedid" id="HC3BD3BS3BA3BO3BT3BG3BM3BN" %) 215 +C;D;S;A;O;T;G;M;N 222 222 )))|((( 223 -== == 217 +(% class="wikigeneratedid" id="H-2" %) 218 + 224 224 )))|((( 225 -== CONF_STATUS will usually be C but may also be D;S;A;O;T;G;M;N depending on the required status and confidentiality reason. See the CL_CONF_STATUS code list for details[[~[3~]>>path:#_ftn3]] == 220 +(% class="wikigeneratedid" id="HCONF_STATUSwillusuallybeCbutmayalsobeD3BS3BA3BO3BT3BG3BM3BNdependingontherequiredstatusandconfidentialityreason.A0SeetheCL_CONF_STATUScodelistfordetails5B35D" %) 221 +CONF_STATUS will usually be C but may also be D;S;A;O;T;G;M;N depending on the required status and confidentiality reason. See the CL_CONF_STATUS code list for details{{footnote}}https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx{{/footnote}} 226 226 ))) 227 -|**Forwarding of confidential data**|N|((( 223 +|((( 224 +**Forwarding of confidential data** 225 +)))|((( 226 +N 227 +)))|((( 228 228 CONF_REDIST: (Observation, Conditional) 229 229 230 230 231 231 )))|CONF_REDIST may represent multiple organisations 232 232 |((( 233 -== Embargo: Privileged access == 233 +(% class="wikigeneratedid" id="HEmbargo:Privilegedaccess" %) 234 +Embargo: Privileged access 234 234 )))|((( 235 -== E == 236 +(% class="wikigeneratedid" id="HE" %) 237 +E 236 236 )))|((( 237 -== EMBARGO_TIME (Observation, Conditional) == 239 +(% class="wikigeneratedid" id="HEMBARGO_TIME28Observation2CConditional29" %) 240 +EMBARGO_TIME (Observation, Conditional) 238 238 239 239 240 240 )))|Only the observations with an EMBARGO_TIME attribute are embargoed. After the embargo time elapses, the data are free for publication (equivalent to F status). 241 241 |((( 242 -== Embargo: Privileged access with forwarding == 245 +(% class="wikigeneratedid" id="HEmbargo:Privilegedaccesswithforwarding" %) 246 +Embargo: Privileged access with forwarding 243 243 )))|((( 244 -== E == 248 +(% class="wikigeneratedid" id="HE-1" %) 249 +E 245 245 )))|((( 246 246 EMBARGO_TIME (Observation, Conditional) 247 247 248 -== CONF_REDIST: (Observation, Conditional) == 253 +(% class="wikigeneratedid" id="HCONF_REDIST:28Observation2CConditional29" %) 254 +CONF_REDIST: (Observation, Conditional) 249 249 )))|((( 250 -== Only the observations with an EMBARGO_TIME attribute are embargoed. After the embargo time elapses, the data are free for publication (equivalent to F status). == 256 +(% class="wikigeneratedid" id="HOnlytheobservationswithanEMBARGO_TIMEattributeareembargoed.Aftertheembargotimeelapses2Cthedataarefreeforpublication28equivalenttoFstatus29." %) 257 +Only the observations with an EMBARGO_TIME attribute are embargoed. After the embargo time elapses, the data are free for publication (equivalent to F status). 251 251 252 -== CONF_REDIST may represent multiple organisations == 259 +(% class="wikigeneratedid" id="HCONF_REDISTmayrepresentmultipleorganisations" %) 260 +CONF_REDIST may represent multiple organisations 253 253 ))) 254 254 |((( 255 -== Embargo: Frontloading == 263 +(% class="wikigeneratedid" id="HEmbargo:Frontloading" %) 264 +Embargo: Frontloading 256 256 )))|((( 257 -== Set to the required confidentiality status after the embargo time elapses. == 266 +(% class="wikigeneratedid" id="HSettotherequiredconfidentialitystatusaftertheembargotimeelapses." %) 267 +Set to the required confidentiality status after the embargo time elapses. 258 258 )))|((( 259 -== <Header\EmbargoDate>: [timestamp] == 269 +(% class="wikigeneratedid" id="H3CHeader5CEmbargoDate3E:5Btimestamp5D" %) 270 +<Header\EmbargoDate>: [timestamp] 260 260 261 261 262 -)))|There is no EMBARGO_TIME attribute as the whole message is embargoed with no privileged access. 273 +)))|((( 274 +There is no EMBARGO_TIME attribute as the whole message is embargoed with no privileged access. 275 +))) 263 263 264 - 265 - 266 - 267 267 ---- 268 268 269 -[[~[1~]>>path:#_ftnref1]] For a full list of confidentiality statuses, see [[https:~~/~~/sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx>>url:https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx]]. 270 - 271 -[[~[2~]>>path:#_ftnref2]] Example: National statistical institute XX reporting data to Eurostat indicates that Eurostat can forward those data to the ECB, IMF and OECD. More complex use case: The reporting organization specifies that Eurostat can forward those data only to the ECB Statistics Department, thus excluding all other organisations as well as all other ECB departments. 272 - 273 -[[~[3~]>>path:#_ftnref3]] [[https:~~/~~/sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx>>url:https://sdmx.org/wp-content/uploads/CL_CONF_STATUS_1_2_2018.docx]] 279 + {{putFootnotes/}}