Interoperability Basis Platform

= 1. Introduction =

This document aims at providing guidelines on how to version SDMX artefacts inspired by "//semantic versioning//", i.e. a formal convention for specifying compatibility between the different versions of a “versionable” artefact (a SDMX artefact that has an associated version number). There are slight differences when compared to semantic versioning regarding the numbering and the definition of the criterion triggering the changes in numbering.

8

9

A three-component versioning system is proposed, with the third component being optional. The criterion for deciding which component is impacted is the severity of the change.

10

11

Versioning is central to SDMX because it guarantees the stability of references to SDMX artefacts. This is of the utmost importance given the sometimes strong dependencies between artefacts, especially in Data Structure Definitions (DSDs).

12

13

The document contains three main recommendations:

14

15

* numbering system and syntax;

16

* types of artefact changes and their versioning impact;

17

* how versioning works for inter-dependent artefacts.

18

19

The document’s appendix contains examples of several types of changes and their versioning impact.

20

21

= 2. Numbering system and syntax =

22

23

The proposed versioning system is based on the Semantic Versioning 2.0 specification{{footnote}}http://www.semver.org{{/footnote}}, namely:

24

25

(% class="wikigeneratedid" id="HMAJOR.MINOR.PATCH" %)

26

MAJOR.MINOR.PATCH{{footnote}}It should be noted that the SDMX standard specifies no limitation as to the number of components in the versioning system. The option proposed here is thus nothing but a recommended convention.{{/footnote}}

27

28

However, as the "patch" component will generally not be used extensively in SDMX, it is proposed to limit the coding to MAJOR.MINOR as long as no patches are implemented. Concretely, this means that version number 2.1.0 will be abridged to 2.1 as long as no patch is implemented. When a patch is implemented, the version number then becomes 2.1.1. At subsequent MAJOR change in the versioning the PATCH component will disappear (2.4.7 → 3.0).

29

30

The most severe change has always precedence over other types of changes. For example, if the MAJOR and MINOR parts of the version number are impacted by changes, only the MAJOR component will be impacted. This means that version 3.2.1 will become 4.0.

31

32

When an artefact is published in production for the first time, the version number of the artefact should be 1.0.

33

34

= 3. Criterion for incrementing the version number =

35

36

The criterion for deciding which component is impacted is the severity of the change, i.e. the possibility of maintaining backward and forward compatibility between the different versions of an artefact.

37

38

== a. Description of backward/forward compatibility ==

39

40

Backward compatibility is defined as: An item (e.g. a data message) that was produced and validated with the previous version of an artefact (e.g. a DSD) can still be successfully validated using the newest version of the same artefact. For example, a data message produced and validated with a DSD version 1.1 is still valid against the same DSD (same id and Agency) upgraded to version 1.2.

41

42

Forward compatibility is defined as: An item (e.g. a data message) that is produced and validated with the new version of an artefact (e.g. a DSD) can also be validated using the previous version of the same artefact. For example, a data message produced and validated with a DSD version 1.1 is also valid against the same DSD (same id and Agency) having version 1.0 (an earlier version).

43

44

Given the syntax specified above, namely MAJOR.MINOR.PATCH, implementers should increment the:

45

46

* MAJOR version when changes are not backward compatible;

47

* MINOR version when changes are backward but not forward compatible;

48

* PATCH version when minor changes (e.g. text clarifications, correction of typos) are both backward and forward compatible.

49

50

== **// //**b. Cost-benefit analysis for a major version change ==

51

52

The cost of imposing a “major” change should be balanced against the benefit of retaining backward compatibility, for example by not deleting codes used in existing data exchanges or by deleting or replacing codes only through a concerted effort of all data exchange partners.

53

54

== c. Synthesis based on the above syntax and criterion ==

55

56

|(% colspan="2" %)**Change Severity**|**Version Impact**|**Description**|**Example**

57

|(% colspan="2" %)**Major**|**+.0**|__**Neither**__ backward __**nor**__ forward compatibility|**1.2 **→**2.0**

58

|(% colspan="2" %)**Minor**|**N.+**|Backward __**but not**__ forward compatibility|**1.0 **→**1.1**

59

|(% colspan="2" %)**Patch**|**N.M.+**|Backward __**and**__ forward compatibility|**1.2 1.**→**2.1**

60

61

=== 4. Types of artefact changes and their versioning impact ===

62

63

As a general rule insignificant changes (e.g. textual clarifications or typos) will result in an increment of the patch component of the versioning system (i.e. N.M.**+**).

64

65

|(% colspan="3" %)**CODE LIST (CL)**

66

|(% style="width:877px" %)**Type of Change**|(% style="width:161px" %)**Impact**|(% style="width:1034px" %)**Comments**

67

|(% style="width:877px" %)(((

68

**Addition into an existing CL of one or more new codes not having the**

69

70

**CodeList:Code:ParentCode attribute**

71

)))|(% style="width:161px" %)**Minor**: **N.,,+,,**^^({{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1034px" %)Data exchanged/disseminated using the old CL can still be exchanged/disseminated using the new CL

72

|(% style="width:877px" %)(((

73

**Addition of one or more new hierarchies represented using the**

74

75

**CodeList:Code:ParentCode attribute (not using the Hierarchical Code List artefact)**

76

)))|(% style="width:161px" %)**Minor**: **N.,,+,,^^(^^**^^{{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1034px" %)Data exchanged/disseminated using the old CL can still be exchanged/disseminated using the new CL as already existing hierarchies still represent the same aggregations

77

|(% style="width:877px" %)**Addition of one or more new codes into existing hierarchies represented using the CodeList:Code:ParentCode attribute (not using the Hierarchical Code List artefact)**|(% style="width:161px" %)**Major**: **+.0**|(% style="width:1034px" %)After the change, the parent code for the changed hierarchy does not represent the same aggregation any more, thus resulting in a break in backward compatibility

78

|(% style="width:877px" %)**Aggregation, disaggregation, reorganisation or removal of one or more codes**|(% style="width:161px" %)**Major**: **+.0**|(% style="width:1034px" %)Data exchanged/disseminated using an old version of the CL can no longer be exchanged/disseminated using the new version of the CL

|(% colspan="3" %)**HIERARCHICAL CODE LIST (HCL)**

83

|**Type of Change**|(% style="width:150px" %)**Impact**|(% style="width:1045px" %)**Comments**

84

|(((

85

**Addition of new hierarchies in the HCL. Existing hierarchies are unaffected**

86

)))|(% style="width:150px" %)**Minor**: **N.,,+,,**^^({{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1045px" %)Data represented using the old HCL can still be represented using the new HCL

87

|**Addition of codes into existing hierarchies in the HCL. Existing hierarchies are thus affected**|(% style="width:150px" %)**Major**: **+.0**|(% style="width:1045px" %)The HCL resulting from this change does not represent the same aggregation any more, thus breaking backward compatibility

88

|**Removal of one or more codes in the HCL or removal of one or more codes in the referenced code lists**|(% style="width:150px" %)**Major**: **+.0**|(% style="width:1045px" %)Data represented using the old HCL can no longer be represented using the new HCL, thus resulting in a break in backward compatibility

89

|**Addition, modification or removal of one or more hierarchical levels**|(% style="width:150px" %)**Major: +.0**|(% style="width:1045px" %)The reorganisation of codes within hierarchies has a significant impact on the code aggregations

|(% colspan="3" %)**CONCEPT SCHEME (CS)**

94

|**Type of change**|**Impact**|**Comments**

95

|**Addition of one or more new concepts in an existing CS**|**Minor**: **N.+**|(((

96

Data exchanged/disseminated using the old version of the

97

98

CS can still be exchanged/disseminated using the new CS

99

)))

100

|**Removal of one or more existing concepts**|**Major: +.0**|Data exchanged/disseminated using the old version of the CS can no longer be exchanged/disseminated using the new version with less concepts

|(% colspan="3" %)**DATA STRUCTURE DEFINITION (DSD)**

105

|**Type of change**|**Impact**|**Comments**

106

|**Addition of a dimension**|**Major**: **+.0**|Adding a new dimension has a strong impact because a dimension represents the identifier of a dataset, thus requiring a remodelling of the data as existing structural validation will fail

107

|**Addition of a mandatory attribute**|**Major**: **+.0**|If the attribute is mandatory, the situation is the same as under point “Addition of a dimension”

108

|**Addition of a conditional attribute**|**Minor**: **N.+**|If the attribute is conditional backward compatibility is maintained

109

|**Removal of a dimension or attribute**|**Major**: **+.0**|Whatever the type of component, the change does not guarantee backward compatibility

110

111

For concrete examples, see the Appendix.

112

113

=== 5. How versioning works for inter-dependent artefacts ===

114

115

This section describes how version changes to inter-dependent or parent/child artefacts affect each other. For example, how a Concept Scheme is affected when one of the Code Lists that it references changes version.

116

117

Some artefacts have references to other artefacts. For example:

118

119

* each of a Concept Scheme’s Concepts may reference a Code List;

120

* a DSD can reference one or more Concept Schemes;

121

* each of a DSD’s Concepts may reference a Code List. (Note that if a Concept-Code List reference exists both in a DSD and a Concept Scheme, the Concept-Code List reference in the DSD overrides the reference in the Concept Scheme);

122

* a Hierarchical Code List references one or more Code Lists whose codes are arranged in the hierarchical structure.

123

124

In the text below, the following concepts will be used:

125

126

* **Parent artefact**: an artefact that contains a reference to another artefact. For example, a Concept Scheme is a parent to a Code List that it references, and the Code List is the child;

127

* **Child artefact**: an artefact that is referenced by another artefact. For example, a Code List is a child of a Concept Scheme that contains a reference to it, and the Concept Scheme is the parent.

128

129

It is important to note that a new version of a child artefact does not automatically trigger a version update of the parent artefact. A version change to the parent artefact is made only if the new version of the child artefact is adopted by the parent artefact.

130

131

==== a. Impact on parent artefact when child artefact version changes ====

132

133

The replacement of a reference with a different reference has the same impact for every artefact.

134

135

|(% colspan="3" %)**ALL ARTEFACTS**

136

|**Type of change**|**Impact**|**Comments**

137

|(((

138

**Replacement of a child artefact having a different version, but same id and**

**Agency**

)))|(((

**The child artefact version change is replicated in the**

**parent artefact**

)))|(((

If a child artefact (e.g. a Code List) has a minor version change, then the parent artefact (e.g. a Concept Scheme) should also have a minor version change.

147

148

If there are several child artefact version changes, the most severe impact is replicated in the parent artefact. For example, if two Code Lists have minor changes, and one Code List has a major change at the same time, the parent Concept Scheme has a major version change

149

)))

150

|(((

151

**Replacement of a referenced child artefact having a**

152

153

**different id or Agency**

154

)))|**The parent artefact version impact depends on the backward/ forward compatibility as shown in the tables above**|Technically, the child artefact is not considered to be related to the previous child artefact. It needs to be checked whether exchange contracts can still be guaranteed (backward/forward compatibility principle)

155

156

==== b. Addition or removal of referenced artefacts ====

157

158

| |(% colspan="2" %)**CONCEPT SCHEME (CS)**

159

|**Type of change**|**Impact**|**Comments**

160

|(((

161

**Addition or removal of a child**

162

163

**Code List**

164

)))|**Minor: N.+**|The child Code Lists in a Data Structure Definition have priority over those referenced in a Concept Scheme. Child Code Lists added to or removed from a Concept Scheme do not have a direct impact on the data exchange. Backward/forward compatibility depends on the way Code Lists are referenced in Data Structure Definitions referencing the concept scheme. This needs to be taken into account when creating a new version of a DSD accordingly

| |(% colspan="2" %)**DATA STRUCTURE DEFINITION (DSD)**

169

|**Type of change**|**Impact**|**Comments**

170

|**Addition or removal of a child Code List**|(((

171

**If same id and Agency, then the child artefact version change is replicated in the parent artefact.**

172

173

**If different id or Agency, impact wil depend on the backward/forward compatibility as shown in the tables above**

174

)))|(((

175

If a child Code List has a minor version change, then the DSD should also have a minor version change.

176

177

If there are several Code List version changes, the most severe impact is replicated in the DSD. For example, if two Code Lists have minor changes, and one Code List has a major change at the same time, the parent DSD has a major version change

178

)))

179

180

=== 6. Appendix - Examples ===

181

182

**Example 1 – Change to a Code List name, for clarification purposes**. **Patch Impact: N.M.+**

183

184

|**Id**|**Old Name**|**New Name**

185

|**CL_ADJUSTMENT**|Adjustment codes|Adjustment code list

186

187

**Example 2** – **Change to a Concept name, for clarification purposes. Patch impact: N.M.+**

188

189

|**Id**|**Old name**|**New name**

190

|**PRODUCT_TO**|Product classification|Product classification (input-output product*product)

191

192

**Example 3** – **Change in the substance of codes. Major impact: +.0**

193

194

|**Id**|**Old name**|**New name**

195

|**CP01115**|Other products|Pizza and quiche

196

197

**Example 4 **- **Aggregation, disaggregation or reorganisation of codes. Major impact**: **+.0**

198

199

|(% colspan="2" %)**AGGREGATION OF EXISTING CODES**

200

|**Old version**|**New version**

201

|**2011** Heifers (female bovine that never calved), live **2012** Cows, live|**2010** Heifers and cows, live

202

|(% colspan="2" %)Codes **2011** and **2012** are fully{{footnote}}i.e. without integration into or combination with another existing code.{{/footnote}} **removed** and replaced with one **brand new** code. In this case there is a many to 1 correspondence between the codes.

|(% colspan="2" %)**DISAGGREGATION OF EXISTING CODES**

207

|**Old version**|**New version**

208

|**1010** Live horses|(((

209

1. Pure bred breeding horses, live

210

1. Other horses, live

211

)))

212

|(% colspan="2" %)Code **1010** is fully **removed** and replaced with two **brand new** codes. In this case there is a 1 to m correspondence between the codes.

|(% colspan="2" %)**REORGANISATION OF EXISTING CODES**

217

|**Old version**|**New version**

218

|(((

219

**3010** Fowls, weighing ≤ 185 g

220

221

**3020** Ducks, , weighing ≤ 185 g

222

223

**3030** Other poultry, weighing ≤ 185 g

224

225

**3040** Fowls, weighing > 185 g

226

227

**3050** Ducks, , weighing > 185 g

228

229

**3060** Other poultry, weighing > 185 g

230

)))|**3025** Poultry, weighing ≤ 175 g **3045** Poultry, weighing > 175 g

231

|(% colspan="2" %)Codes **3010**, **3020**, **3030**, **3040**, **3050** and **3060** are fully removed and replaced with two brand new codes; furthermore the criterion for the classification used in the old version has been changed in the new version (185 g criterion versus 175 g criterion), so that it is not possible to exactly aggregate the codes from the old version to the codes of the new version (e.g. a part of **3010** goes to **3025**, another part to **3045**). In this case there is a m to n correspondence between the two sets of codes

232

233

**Example 5 – Changes to hierarchies in a Code List. Major impact: +.0**

234

235

|(% colspan="2" %)**ADDING A NEW CODE IN AN EXISTING HIERARCHY – CODE LIST**

236

|**Old version**|**New version**

237

|• 0213 - Beer o02131 - Lager beer o02132 - Other alcoholic beer|• 0213 - Beer o02131 - Lager beer o 02132 - Other alcoholic beer o **02133 - Low and non-alcoholic beer**

238

|(% colspan="2" %)Code 02133 has been added to hierarchy 0213

239

240

**Example 6 – Changes to hierarchies in a Hierarchical Code List. Major impact: +.0**

241

242

|(% colspan="2" %)**ADDING A NEW CODE IN AN EXISTING HIERARCHY – HIERARCHICAL CODE LIST**

243

|**Old version**|**New version**

244

|(((

245

• A1 - World (codelist ref. ECB@CL_AREAS@1.0) o E1 - Europe (ECB@CL_COUNTRIES@1.0)

246

247

ES - Spain FR - France

GR - Greece

IT - Italy o E4 - Africaetc.

252

)))|(((

253

• A1=World (codelist ref. ECB@CL_AREAS@1.0) o E1 =Europe (ECB@CL_COUNTRIES@1.0)

254

255

ES = Spain FR = FranceGR = Greece

IT = Italy

**DE= Germany**

o E4 =Africaetc.

)))

|(% colspan="2" %)The id of the hierarchical codes are assumed to be equal to those of the code lists referenced. The code DE has been added to hierarchy E1

264

265

**Example 7.1 – Dependencies between artefacts: Concept Scheme and Code List. Minor impact: N.+**

**New**

**version**

)))

|**CS_TRADE:Concept Scheme: References CL_OBS_STATUS v1.0 above**|(((

274

Adoption of new code X

275

276

**//Change type//**: Replacement of a child artefact having a different version, but the same id and

Agency

)))|(((

**Minor: N.+**

The child version impact is replicated in the parent

artefact

)))|**2.0**|**2.1**

**Example 7.2 – Dependencies between artefacts: Concept Scheme and Code List. Major impact: +.0**

**New**

**version**

)))

|(((

**CS_TRADE:Concept**

**Scheme:References**

299

300

**CL_OBS_STATUS v1.0 above**

301

)))|(((

302

Adoption of new CL_OBS_STATUS without U.

303

304

**//Change type//**: Replacement of a child artefact having a different version, but the same id and

Agency

)))|(((

**Major: +.0**

The child version impact is replicated in the parent artefact.

311

)))|**2.0**|**3.0**

312

313

**Example 7.3 – Dependencies between artefacts: Concept Scheme and Code List. Variable impact (see below)**

**New**

**version**

)))

|(% rowspan="3" %)**CL_XYZ: Code List**|(((

321

a) Maintenance agency changes from A to B for governance reasons.

322

323

Nothing else changes in the code list.

324

)))|(% rowspan="3" %)**New artefact**|(% rowspan="3" %)(((

**CL_XYZ**

**(Agency A)**

)))|(% rowspan="3" %)(((

**CL_XYZ**

**(Agency B)**

**(new maintenance agency)**

334

)))

335

|b) Maintenance agency changes from A to B and at the same time new codes are added

336

|c) Maintenance agency changes from A to B. Since B has different coding rules, the code list itself changes as well.

337

|(% rowspan="3" %)(((

338

**CS_TRADE: Concept Scheme: References**

339

340

**CL_XYZ (Agency A) **

341

)))|(% rowspan="3" %)(((

342

Replacement of a child artefact having a different Agency.

343

344

CL_XYZ (Agency A) changes to CL_XYZ (Agency B).

345

)))|(((

346

**Case a): Patch: N.M.+**

347

348

There is no impact on data exchange

349

)))|**2.0**|**2.0.1**

350

|(((

351

**Case b): Minor: N. +**

352

353

The impact is the same as a new minor version of the code list

354

)))|**2.0**|**2.1**

355

|(((

356

**Case c) Major: +.0**

357

358

The impact is the same as a new major version of the code list.

359

)))|**2.0**|**3.0**

360

361

**Example 7.4 – Dependencies between artefacts: Concept Scheme and DSD. Variable impact (see below)**

|(((

**TRADE: Data Structure Definition:**

367

368

**references Concepts C1 and C2**

)))|(((

None

Concept C3 is not used

373

)))|**None**|**1.0**|**1.0**

374

|** **| |** **|** **|** **

375

376

|(((

377

**TRADE: Data Structure Definition:**

378

379

**references Concepts C1 and C2**

)))|(((

None

Concept C3 is not used

384

)))|**None**|**1.0**|**1.0**

|(% colspan="5" %)

|(((

**TRADE: Data Structure Definition:**

389

390

**references Concepts C1 and C2**

391

)))|None concept C3 is not used|**None**|**1.0**|**1.0**

392

|(% colspan="5" %)**Remark: **Once a new version of the DSD is needed for some other reasons (e.g. a change in a code list), it is recommended to update all concept references to the newest available concept scheme if possible: i.e. DSD version 1.1 would then update its concept scheme references from 1.4 to 2.0.

|(% colspan="5" %)

|(((

**TRADE: Data Structure Definition:**

397

398

**references Concepts C1 and C2**

399

)))|Correction should be taken into account, concept C2 is used|(((

**Patch: N.M.+**

**//or//**

**None**

)))|**1.0**|(((

**1.0.1 //or//**

**1.0**

)))

|(% colspan="5" %)**Remark: **Since the change of a typo in a Concept of the Concept Scheme does not have a direct impact on the DSD itself (the link is by reference), there is strictly speaking no need to update the DSD. Both DSDs (1.0 and 1.0.1) will have exactly the same syntax. However, if maintainers want to highlight the correction for users of the DSD or for some other reason the DSD is updated anyway; it should reference the newer Concept Scheme.

**~ **

----

Wiki source code of Guidelines on the Versioning of SDMX Artefacts