Interoperability Basis Platform

= 1. Introduction =

This document aims at providing guidelines on how to version SDMX artefacts inspired by "//semantic versioning//", i.e. a formal convention for specifying compatibility between the different versions of a “versionable” artefact (a SDMX artefact that has an associated version number). There are slight differences when compared to semantic versioning regarding the numbering and the definition of the criterion triggering the changes in numbering.

8

9

A three-component versioning system is proposed, with the third component being optional. The criterion for deciding which component is impacted is the severity of the change.

10

11

Versioning is central to SDMX because it guarantees the stability of references to SDMX artefacts. This is of the utmost importance given the sometimes strong dependencies between artefacts, especially in Data Structure Definitions (DSDs).

12

13

The document contains three main recommendations:

14

15

* numbering system and syntax;

16

* types of artefact changes and their versioning impact;

17

* how versioning works for inter-dependent artefacts.

18

19

The document’s appendix contains examples of several types of changes and their versioning impact.

20

21

= 2. Numbering system and syntax =

22

23

The proposed versioning system is based on the Semantic Versioning 2.0 specification{{footnote}}http://www.semver.org{{/footnote}}, namely:

24

25

(% class="wikigeneratedid" id="HMAJOR.MINOR.PATCH" %)

26

MAJOR.MINOR.PATCH{{footnote}}It should be noted that the SDMX standard specifies no limitation as to the number of components in the versioning system. The option proposed here is thus nothing but a recommended convention.{{/footnote}}

27

28

However, as the "patch" component will generally not be used extensively in SDMX, it is proposed to limit the coding to MAJOR.MINOR as long as no patches are implemented. Concretely, this means that version number 2.1.0 will be abridged to 2.1 as long as no patch is implemented. When a patch is implemented, the version number then becomes 2.1.1. At subsequent MAJOR change in the versioning the PATCH component will disappear (2.4.7 → 3.0).

29

30

The most severe change has always precedence over other types of changes. For example, if the MAJOR and MINOR parts of the version number are impacted by changes, only the MAJOR component will be impacted. This means that version 3.2.1 will become 4.0.

31

32

When an artefact is published in production for the first time, the version number of the artefact should be 1.0.

33

34

= 3. Criterion for incrementing the version number =

35

36

The criterion for deciding which component is impacted is the severity of the change, i.e. the possibility of maintaining backward and forward compatibility between the different versions of an artefact.

37

38

== a. Description of backward/forward compatibility ==

39

40

Backward compatibility is defined as: An item (e.g. a data message) that was produced and validated with the previous version of an artefact (e.g. a DSD) can still be successfully validated using the newest version of the same artefact. For example, a data message produced and validated with a DSD version 1.1 is still valid against the same DSD (same id and Agency) upgraded to version 1.2.

41

42

Forward compatibility is defined as: An item (e.g. a data message) that is produced and validated with the new version of an artefact (e.g. a DSD) can also be validated using the previous version of the same artefact. For example, a data message produced and validated with a DSD version 1.1 is also valid against the same DSD (same id and Agency) having version 1.0 (an earlier version).

43

44

Given the syntax specified above, namely MAJOR.MINOR.PATCH, implementers should increment the:

45

46

* MAJOR version when changes are not backward compatible;

47

* MINOR version when changes are backward but not forward compatible;

48

* PATCH version when minor changes (e.g. text clarifications, correction of typos) are both backward and forward compatible.

49

50

== **// //**b. Cost-benefit analysis for a major version change ==

51

52

The cost of imposing a “major” change should be balanced against the benefit of retaining backward compatibility, for example by not deleting codes used in existing data exchanges or by deleting or replacing codes only through a concerted effort of all data exchange partners.

53

54

== c. Synthesis based on the above syntax and criterion ==

55

56

|(% colspan="2" %)**Change Severity**|**Version Impact**|**Description**|**Example**

57

|(% colspan="2" %)**Major**|**+.0**|__**Neither**__ backward __**nor**__ forward compatibility|**1.2 **→**2.0**

58

|(% colspan="2" %)**Minor**|**N.+**|Backward __**but not**__ forward compatibility|**1.0 **→**1.1**

59

|(% colspan="2" %)**Patch**|**N.M.+**|Backward __**and**__ forward compatibility|**1.2 1.**→**2.1**

60

61

=== 4. Types of artefact changes and their versioning impact ===

62

63

As a general rule insignificant changes (e.g. textual clarifications or typos) will result in an increment of the patch component of the versioning system (i.e. N.M.**+**).

64

65

|(% colspan="3" %)**CODE LIST (CL)**

66

|(% style="width:877px" %)**Type of Change**|(% style="width:161px" %)**Impact**|(% style="width:1034px" %)**Comments**

67

|(% style="width:877px" %)(((

68

**Addition into an existing CL of one or more new codes not having the**

69

70

**CodeList:Code:ParentCode attribute**

71

)))|(% style="width:161px" %)**Minor**: **N.,,+,,**^^({{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1034px" %)Data exchanged/disseminated using the old CL can still be exchanged/disseminated using the new CL

72

|(% style="width:877px" %)(((

73

**Addition of one or more new hierarchies represented using the**

74

75

**CodeList:Code:ParentCode attribute (not using the Hierarchical Code List artefact)**

76

)))|(% style="width:161px" %)**Minor**: **N.,,+,,^^(^^**^^{{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1034px" %)Data exchanged/disseminated using the old CL can still be exchanged/disseminated using the new CL as already existing hierarchies still represent the same aggregations

77

|(% style="width:877px" %)**Addition of one or more new codes into existing hierarchies represented using the CodeList:Code:ParentCode attribute (not using the Hierarchical Code List artefact)**|(% style="width:161px" %)**Major**: **+.0**|(% style="width:1034px" %)After the change, the parent code for the changed hierarchy does not represent the same aggregation any more, thus resulting in a break in backward compatibility

78

|(% style="width:877px" %)**Aggregation, disaggregation, reorganisation or removal of one or more codes**|(% style="width:161px" %)**Major**: **+.0**|(% style="width:1034px" %)Data exchanged/disseminated using an old version of the CL can no longer be exchanged/disseminated using the new version of the CL

|(% colspan="3" %)**HIERARCHICAL CODE LIST (HCL)**

83

|**Type of Change**|(% style="width:150px" %)**Impact**|(% style="width:1045px" %)**Comments**

84

|(((

85

**Addition of new hierarchies in the HCL. Existing hierarchies are unaffected**

86

)))|(% style="width:150px" %)**Minor**: **N.,,+,,**^^({{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1045px" %)Data represented using the old HCL can still be represented using the new HCL

87

|**Addition of codes into existing hierarchies in the HCL. Existing hierarchies are thus affected**|(% style="width:150px" %)**Major**: **+.0**|(% style="width:1045px" %)The HCL resulting from this change does not represent the same aggregation any more, thus breaking backward compatibility

88

|**Removal of one or more codes in the HCL or removal of one or more codes in the referenced code lists**|(% style="width:150px" %)**Major**: **+.0**|(% style="width:1045px" %)Data represented using the old HCL can no longer be represented using the new HCL, thus resulting in a break in backward compatibility

89

|**Addition, modification or removal of one or more hierarchical levels**|(% style="width:150px" %)**Major: +.0**|(% style="width:1045px" %)The reorganisation of codes within hierarchies has a significant impact on the code aggregations

90

91

|(% colspan="3" %)**CONCEPT SCHEME (CS)**

92

|(% style="width:874px" %)**Type of change**|(% style="width:154px" %)**Impact**|(% style="width:1044px" %)**Comments**

93

|(% style="width:874px" %)**Addition of one or more new concepts in an existing CS**|(% style="width:154px" %)**Minor**: **N.+**|(% style="width:1044px" %)(((

94

Data exchanged/disseminated using the old version of the CS can still be exchanged/disseminated using the new CS

95

)))

96

|(% style="width:874px" %)**Removal of one or more existing concepts**|(% style="width:154px" %)**Major: +.0**|(% style="width:1044px" %)Data exchanged/disseminated using the old version of the CS can no longer be exchanged/disseminated using the new version with less concepts

97

98

|(% colspan="3" %)**DATA STRUCTURE DEFINITION (DSD)**

99

|(% style="width:868px" %)**Type of change**|(% style="width:159px" %)**Impact**|(% style="width:1045px" %)**Comments**

100

|(% style="width:868px" %)**Addition of a dimension**|(% style="width:159px" %)**Major**: **+.0**|(% style="width:1045px" %)Adding a new dimension has a strong impact because a dimension represents the identifier of a dataset, thus requiring a remodelling of the data as existing structural validation will fail

101

|(% style="width:868px" %)**Addition of a mandatory attribute**|(% style="width:159px" %)**Major**: **+.0**|(% style="width:1045px" %)If the attribute is mandatory, the situation is the same as under point “Addition of a dimension”

102

|(% style="width:868px" %)**Addition of a conditional attribute**|(% style="width:159px" %)**Minor**: **N.+**|(% style="width:1045px" %)If the attribute is conditional backward compatibility is maintained

103

|(% style="width:868px" %)**Removal of a dimension or attribute**|(% style="width:159px" %)**Major**: **+.0**|(% style="width:1045px" %)Whatever the type of component, the change does not guarantee backward compatibility

104

105

For concrete examples, see the Appendix.

106

107

= 5. How versioning works for inter-dependent artefacts =

108

109

This section describes how version changes to inter-dependent or parent/child artefacts affect each other. For example, how a Concept Scheme is affected when one of the Code Lists that it references changes version.

110

111

Some artefacts have references to other artefacts. For example:

112

113

* each of a Concept Scheme’s Concepts may reference a Code List;

114

* a DSD can reference one or more Concept Schemes;

115

* each of a DSD’s Concepts may reference a Code List. (Note that if a Concept-Code List reference exists both in a DSD and a Concept Scheme, the Concept-Code List reference in the DSD overrides the reference in the Concept Scheme);

116

* a Hierarchical Code List references one or more Code Lists whose codes are arranged in the hierarchical structure.

117

118

In the text below, the following concepts will be used:

119

120

* **Parent artefact**: an artefact that contains a reference to another artefact. For example, a Concept Scheme is a parent to a Code List that it references, and the Code List is the child;

121

* **Child artefact**: an artefact that is referenced by another artefact. For example, a Code List is a child of a Concept Scheme that contains a reference to it, and the Concept Scheme is the parent.

122

123

It is important to note that a new version of a child artefact does not automatically trigger a version update of the parent artefact. A version change to the parent artefact is made only if the new version of the child artefact is adopted by the parent artefact.

124

125

==== a. Impact on parent artefact when child artefact version changes ====

126

127

The replacement of a reference with a different reference has the same impact for every artefact.

128

129

|(% colspan="3" %)**ALL ARTEFACTS**

130

|(% style="width:492px" %)**Type of change**|(% style="width:441px" %)**Impact**|**Comments**

131

|(% style="width:492px" %)(((

132

**Replacement of a child artefact having a different version, but same id and Agency**

133

)))|(% style="width:441px" %)(((

134

**The child artefact version change is replicated in the parent artefact**

135

)))|(((

136

If a child artefact (e.g. a Code List) has a minor version change, then the parent artefact (e.g. a Concept Scheme) should also have a minor version change.

137

If there are several child artefact version changes, the most severe impact is replicated in the parent artefact. For example, if two Code Lists have minor changes, and one Code List has a major change at the same time, the parent Concept Scheme has a major version change

138

)))

139

|(% style="width:492px" %)(((

140

**Replacement of a referenced child artefact having a different id or Agency**

141

)))|(% style="width:441px" %)**The parent artefact version impact depends on the backward/ forward compatibility as shown in the tables above**|Technically, the child artefact is not considered to be related to the previous child artefact. It needs to be checked whether exchange contracts can still be guaranteed (backward/forward compatibility principle)

142

143

==== b. Addition or removal of referenced artefacts ====

144

145

|(% colspan="3" style="width:876px" %)**CONCEPT SCHEME (CS)**

146

|(% style="width:845px" %)**Type of change**|(% style="width:156px" %)**Impact**|(% style="width:1071px" %)**Comments**

147

|(% style="width:845px" %)(((

148

**Addition or removal of a child Code List**

149

)))|(% style="width:156px" %)**Minor: N.+**|(% style="width:1071px" %)The child Code Lists in a Data Structure Definition have priority over those referenced in a Concept Scheme. Child Code Lists added to or removed from a Concept Scheme do not have a direct impact on the data exchange. Backward/forward compatibility depends on the way Code Lists are referenced in Data Structure Definitions referencing the concept scheme. This needs to be taken into account when creating a new version of a DSD accordingly

|(% colspan="3" %)**DATA STRUCTURE DEFINITION (DSD)**

154

|**Type of change**|**Impact**|**Comments**

155

|**Addition or removal of a child Code List**|(((

156

**If same id and Agency, then the child artefact version change is replicated in the parent artefact.

157

If different id or Agency, impact wil depend on the backward/forward compatibility as shown in the tables above**

158

)))|(((

159

If a child Code List has a minor version change, then the DSD should also have a minor version change.

160

If there are several Code List version changes, the most severe impact is replicated in the DSD. For example, if two Code Lists have minor changes, and one Code List has a major change at the same time, the parent DSD has a major version change

161

)))

162

163

= 6. Appendix - Examples =

164

165

**Example 1 – Change to a Code List name, for clarification purposes**. **Patch Impact: N.M.+**

166

167

|**Id**|**Old Name**|**New Name**

168

|**CL_ADJUSTMENT**|Adjustment codes|Adjustment code list

169

170

**Example 2** – **Change to a Concept name, for clarification purposes. Patch impact: N.M.+**

171

172

|**Id**|**Old name**|**New name**

173

|**PRODUCT_TO**|Product classification|Product classification (input-output product*product)

174

175

**Example 3** – **Change in the substance of codes. Major impact: +.0**

176

177

|**Id**|**Old name**|**New name**

178

|**CP01115**|Other products|Pizza and quiche

179

180

**Example 4 **- **Aggregation, disaggregation or reorganisation of codes. Major impact**: **+.0**

181

182

|(% colspan="2" %)**AGGREGATION OF EXISTING CODES**

183

|**Old version**|**New version**

184

|**2011** Heifers (female bovine that never calved), live

185

**2012** Cows, live|**2010** Heifers and cows, live

186

|(% colspan="2" %)Codes **2011** and **2012** are fully{{footnote}}i.e. without integration into or combination with another existing code.{{/footnote}} __**removed**__ and replaced with one __**brand new**__ code. In this case there is a many to 1 correspondence between the codes.

187

188

|(% colspan="2" %)**DISAGGREGATION OF EXISTING CODES**

189

|**Old version**|**New version**

190

|**1010** Live horses|(((

191

1011 Pure bred breeding horses, live

192

1012 Other horses, live

193

)))

194

|(% colspan="2" %)Code **1010** is fully __**removed**__ and replaced with two __**brand new**__ codes. In this case there is a 1 to m correspondence between the codes.

195

196

|(% colspan="2" %)**REORGANISATION OF EXISTING CODES**

197

|**Old version**|**New version**

198

|(((

199

**3010** Fowls, weighing ≤ 185 g

200

**3020** Ducks, , weighing ≤ 185 g

201

**3030** Other poultry, weighing ≤ 185 g

202

**3040** Fowls, weighing > 185 g

203

**3050** Ducks, , weighing > 185 g

204

**3060** Other poultry, weighing > 185 g

205

)))|**3025** Poultry, weighing ≤ 175 g

206

**3045** Poultry, weighing > 175 g

207

|(% colspan="2" %)Codes **3010**, **3020**, **3030**, **3040**, **3050** and **3060** are fully removed and replaced with two brand new codes; furthermore the criterion for the classification used in the old version has been changed in the new version (185 g criterion versus 175 g criterion), so that it is not possible to exactly aggregate the codes from the old version to the codes of the new version (e.g. a part of **3010** goes to **3025**, another part to **3045**). In this case there is a m to n correspondence between the two sets of codes

208

209

**Example 5 – Changes to hierarchies in a Code List. Major impact: +.0**

210

211

|(% colspan="2" %)**ADDING A NEW CODE IN AN EXISTING HIERARCHY – CODE LIST**

212

|**Old version**|**New version**

213

|(((

214

* 0213 - Beer

215

** 02131 - Lager beer

216

** 02132 - Other alcoholic beer

217

)))|(((

218

* 0213 - Beer

219

** 02131 - Lager beer

220

** 02132 - Other alcoholic beer

221

* **02133 - Low and non-alcoholic beer**

222

)))

223

|(% colspan="2" %)Code 02133 has been added to hierarchy 0213

224

225

**Example 6 – Changes to hierarchies in a Hierarchical Code List. Major impact: +.0**

226

227

|(% colspan="2" %)**ADDING A NEW CODE IN AN EXISTING HIERARCHY – HIERARCHICAL CODE LIST**

228

|**Old version**|**New version**

229

|(((

230

* A1 - World (codelist ref. ECB@CL_AREAS@1.0)

231

** E1 - Europe (ECB@CL_COUNTRIES@1.0)

232

*** ES - Spain FR - France

*** GR - Greece

*** IT - Italy

** E4 - Africa

*** etc.

)))|(((

* A1=World (codelist ref. ECB@CL_AREAS@1.0)

239

** E1 =Europe (ECB@CL_COUNTRIES@1.0)

240

** ES = Spain FR = FranceGR = Greece

** IT = Italy

** **DE= Germany**

* E4 =Africa

** etc.

)))

|(% colspan="2" %)The id of the hierarchical codes are assumed to be equal to those of the code lists referenced. The code DE has been added to hierarchy E1

247

248

**Example 7.1 – Dependencies between artefacts: Concept Scheme and Code List. Minor impact: N.+**

**New version**

)))

|(% style="width:492px" %)**CS_TRADE:Concept Scheme: References CL_OBS_STATUS v1.0 above**|(% style="width:732px" %)(((

255

Adoption of new code X

256

257

**//Change type//**: Replacement of a child artefact having a different version, but the same id and Agency

258

)))|(% style="width:558px" %)(((

259

**Minor: N.+**

260

The child version impact is replicated in the parent artefact

261

)))|(% style="width:139px" %)**2.0**|(% style="width:151px" %)**2.1**

262

263

**Example 7.2 – Dependencies between artefacts: Concept Scheme and Code List. Major impact: +.0**

**New version**

)))

|(% style="width:489px" %)(((

270

**CS_TRADE:Concept**

271

272

**Scheme:References**

273

274

**CL_OBS_STATUS v1.0 above**

275

)))|(% style="width:734px" %)(((

276

Adoption of new CL_OBS_STATUS without U.

277

278

**//Change type//**: Replacement of a child artefact having a different version, but the same id and

279

280

Agency

281

)))|(% style="width:562px" %)(((

282

**Major: +.0**

283

284

The child version impact is replicated in the parent artefact.

285

)))|**2.0**|**3.0**

286

287

**Example 7.3 – Dependencies between artefacts: Concept Scheme and Code List. Variable impact (see below)**

**New version**

)))

|(% rowspan="3" style="width:489px" %)**CL_XYZ: Code List**|(% style="width:736px" %)(((

293

a) Maintenance agency changes from A to B for governance reasons.

294

Nothing else changes in the code list.

295

)))|(% rowspan="3" style="width:555px" %)**New artefact**|(% rowspan="3" style="width:146px" %)(((

**CL_XYZ**

**(Agency A)**

)))|(% rowspan="3" style="width:146px" %)(((

**CL_XYZ**

**(Agency B)

(new maintenance agency)**

304

)))

305

|(% style="width:736px" %)b) Maintenance agency changes from A to B and at the same time new codes are added

306

|(% style="width:736px" %)c) Maintenance agency changes from A to B. Since B has different coding rules, the code list itself changes as well.

307

|(% rowspan="3" style="width:489px" %)(((

308

**CS_TRADE: Concept Scheme: References CL_XYZ (Agency A) **

309

)))|(% rowspan="3" style="width:736px" %)(((

310

Replacement of a child artefact having a different Agency.

311

CL_XYZ (Agency A) changes to CL_XYZ (Agency B).

312

)))|(% style="width:555px" %)(((

313

**Case a): Patch: N.M.+**

314

There is no impact on data exchange

315

)))|(% style="width:146px" %)**2.0**|(% style="width:146px" %)**2.0.1**

316

|(% style="width:555px" %)(((

317

**Case b): Minor: N. +**

318

The impact is the same as a new minor version of the code list

319

)))|(% style="width:146px" %)**2.0**|(% style="width:146px" %)**2.1**

320

|(% style="width:555px" %)(((

321

**Case c) Major: +.0**

322

The impact is the same as a new major version of the code list.

323

)))|(% style="width:146px" %)**2.0**|(% style="width:146px" %)**3.0**

324

325

**Example 7.4 – Dependencies between artefacts: Concept Scheme and DSD. Variable impact (see below)**

|(((

**TRADE: Data Structure Definition:

331

references Concepts C1 and C2**

332

)))|(((

333

None

334

Concept C3 is not used

335

)))|**None**|**1.0**|**1.0**

336

|** **| |** **|** **|** **

337

338

|(((

339

**TRADE: Data Structure Definition:

340

references Concepts C1 and C2**

341

)))|(((

342

None

343

Concept C3 is not used

344

)))|**None**|**1.0**|**1.0**

|(% colspan="5" %)

|(((

**TRADE: Data Structure Definition:

349

references Concepts C1 and C2**

350

)))|None concept C3 is not used|**None**|**1.0**|**1.0**

351

|(% colspan="5" %)**Remark: **Once a new version of the DSD is needed for some other reasons (e.g. a change in a code list), it is recommended to update all concept references to the newest available concept scheme if possible: i.e. DSD version 1.1 would then update its concept scheme references from 1.4 to 2.0.

|(% colspan="5" %)

|(((

**TRADE: Data Structure Definition:

356

references Concepts C1 and C2**

357

)))|Correction should be taken into account, concept C2 is used|(((

**Patch: N.M.+

//or //**

**None**

)))|**1.0**|(((

**1.0.1 //or//**

**1.0**

)))

|(% colspan="5" %)**Remark: **Since the change of a typo in a Concept of the Concept Scheme does not have a direct impact on the DSD itself (the link is by reference), there is strictly speaking no need to update the DSD. Both DSDs (1.0 and 1.0.1) will have exactly the same syntax. However, if maintainers want to highlight the correction for users of the DSD or for some other reason the DSD is updated anyway; it should reference the newer Concept Scheme.

----

Wiki source code of Guidelines on the Versioning of SDMX Artefacts