Interoperability Basis Platform

= 1. Introduction =

This document aims at providing guidelines on how to version SDMX artefacts inspired by "//semantic versioning//", i.e. a formal convention for specifying compatibility between the different versions of a “versionable” artefact (a SDMX artefact that has an associated version number). There are slight differences when compared to semantic versioning regarding the numbering and the definition of the criterion triggering the changes in numbering.

8

9

A three-component versioning system is proposed, with the third component being optional. The criterion for deciding which component is impacted is the severity of the change.

10

11

Versioning is central to SDMX because it guarantees the stability of references to SDMX artefacts. This is of the utmost importance given the sometimes strong dependencies between artefacts, especially in Data Structure Definitions (DSDs).

12

13

The document contains three main recommendations:

14

15

* numbering system and syntax;

16

* types of artefact changes and their versioning impact;

17

* how versioning works for inter-dependent artefacts.

18

19

The document’s appendix contains examples of several types of changes and their versioning impact.

20

21

= 2. Numbering system and syntax =

22

23

The proposed versioning system is based on the Semantic Versioning 2.0 specification{{footnote}}http://www.semver.org{{/footnote}}, namely:

24

25

(% class="wikigeneratedid" id="HMAJOR.MINOR.PATCH" %)

26

MAJOR.MINOR.PATCH{{footnote}}It should be noted that the SDMX standard specifies no limitation as to the number of components in the versioning system. The option proposed here is thus nothing but a recommended convention.{{/footnote}}

27

28

However, as the "patch" component will generally not be used extensively in SDMX, it is proposed to limit the coding to MAJOR.MINOR as long as no patches are implemented. Concretely, this means that version number 2.1.0 will be abridged to 2.1 as long as no patch is implemented. When a patch is implemented, the version number then becomes 2.1.1. At subsequent MAJOR change in the versioning the PATCH component will disappear (2.4.7 → 3.0).

29

30

The most severe change has always precedence over other types of changes. For example, if the MAJOR and MINOR parts of the version number are impacted by changes, only the MAJOR component will be impacted. This means that version 3.2.1 will become 4.0.

31

32

When an artefact is published in production for the first time, the version number of the artefact should be 1.0.

33

34

= 3. Criterion for incrementing the version number =

35

36

The criterion for deciding which component is impacted is the severity of the change, i.e. the possibility of maintaining backward and forward compatibility between the different versions of an artefact.

37

38

== a. Description of backward/forward compatibility ==

39

40

Backward compatibility is defined as: An item (e.g. a data message) that was produced and validated with the previous version of an artefact (e.g. a DSD) can still be successfully validated using the newest version of the same artefact. For example, a data message produced and validated with a DSD version 1.1 is still valid against the same DSD (same id and Agency) upgraded to version 1.2.

41

42

Forward compatibility is defined as: An item (e.g. a data message) that is produced and validated with the new version of an artefact (e.g. a DSD) can also be validated using the previous version of the same artefact. For example, a data message produced and validated with a DSD version 1.1 is also valid against the same DSD (same id and Agency) having version 1.0 (an earlier version).

43

44

Given the syntax specified above, namely MAJOR.MINOR.PATCH, implementers should increment the:

45

46

* MAJOR version when changes are not backward compatible;

47

* MINOR version when changes are backward but not forward compatible;

48

* PATCH version when minor changes (e.g. text clarifications, correction of typos) are both backward and forward compatible.

49

50

== **// //**b. Cost-benefit analysis for a major version change ==

51

52

The cost of imposing a “major” change should be balanced against the benefit of retaining backward compatibility, for example by not deleting codes used in existing data exchanges or by deleting or replacing codes only through a concerted effort of all data exchange partners.

53

54

== c. Synthesis based on the above syntax and criterion ==

55

56

|(% colspan="2" %)**Change Severity**|**Version Impact**|**Description**|**Example**

57

|(% colspan="2" %)**Major**|**+.0**|__**Neither**__ backward __**nor**__ forward compatibility|**1.2 **→**2.0**

58

|(% colspan="2" %)**Minor**|**N.+**|Backward __**but not**__ forward compatibility|**1.0 **→**1.1**

59

|(% colspan="2" %)**Patch**|**N.M.+**|Backward __**and**__ forward compatibility|**1.2 1.**→**2.1**

60

61

=== 4. Types of artefact changes and their versioning impact ===

62

63

As a general rule insignificant changes (e.g. textual clarifications or typos) will result in an increment of the patch component of the versioning system (i.e. N.M.**+**).

64

65

|(% colspan="3" %)**CODE LIST (CL)**

66

|(% style="width:877px" %)**Type of Change**|(% style="width:161px" %)**Impact**|(% style="width:1034px" %)**Comments**

67

|(% style="width:877px" %)(((

68

**Addition into an existing CL of one or more new codes not having the**

69

70

**CodeList:Code:ParentCode attribute**

71

)))|(% style="width:161px" %)**Minor**: **N.,,+,,**^^({{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1034px" %)Data exchanged/disseminated using the old CL can still be exchanged/disseminated using the new CL

72

|(% style="width:877px" %)(((

73

**Addition of one or more new hierarchies represented using the**

74

75

**CodeList:Code:ParentCode attribute (not using the Hierarchical Code List artefact)**

76

)))|(% style="width:161px" %)**Minor**: **N.,,+,,^^(^^**^^{{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1034px" %)Data exchanged/disseminated using the old CL can still be exchanged/disseminated using the new CL as already existing hierarchies still represent the same aggregations

77

|(% style="width:877px" %)**Addition of one or more new codes into existing hierarchies represented using the CodeList:Code:ParentCode attribute (not using the Hierarchical Code List artefact)**|(% style="width:161px" %)**Major**: **+.0**|(% style="width:1034px" %)After the change, the parent code for the changed hierarchy does not represent the same aggregation any more, thus resulting in a break in backward compatibility

78

|(% style="width:877px" %)**Aggregation, disaggregation, reorganisation or removal of one or more codes**|(% style="width:161px" %)**Major**: **+.0**|(% style="width:1034px" %)Data exchanged/disseminated using an old version of the CL can no longer be exchanged/disseminated using the new version of the CL

|(% colspan="3" %)**HIERARCHICAL CODE LIST (HCL)**

83

|**Type of Change**|(% style="width:150px" %)**Impact**|(% style="width:1045px" %)**Comments**

84

|(((

85

**Addition of new hierarchies in the HCL. Existing hierarchies are unaffected**

86

)))|(% style="width:150px" %)**Minor**: **N.,,+,,**^^({{footnote}}The overall impact on compatibility should be assessed when there are several “minor” version impact changes. For example, it may be that the effect of adding several new Code List or HCL codes results in an implicit change in the meaning of existing Code List or HCL codes which may not be completely backward compatible, therefore (depending on the analysis) the overall version impact may be “Major +.0”.{{/footnote}})^^|(% style="width:1045px" %)Data represented using the old HCL can still be represented using the new HCL

87

|**Addition of codes into existing hierarchies in the HCL. Existing hierarchies are thus affected**|(% style="width:150px" %)**Major**: **+.0**|(% style="width:1045px" %)The HCL resulting from this change does not represent the same aggregation any more, thus breaking backward compatibility

88

|**Removal of one or more codes in the HCL or removal of one or more codes in the referenced code lists**|(% style="width:150px" %)**Major**: **+.0**|(% style="width:1045px" %)Data represented using the old HCL can no longer be represented using the new HCL, thus resulting in a break in backward compatibility

89

|**Addition, modification or removal of one or more hierarchical levels**|(% style="width:150px" %)**Major: +.0**|(% style="width:1045px" %)The reorganisation of codes within hierarchies has a significant impact on the code aggregations

90

91

|(% colspan="3" %)**CONCEPT SCHEME (CS)**

92

|(% style="width:874px" %)**Type of change**|(% style="width:154px" %)**Impact**|(% style="width:1044px" %)**Comments**

93

|(% style="width:874px" %)**Addition of one or more new concepts in an existing CS**|(% style="width:154px" %)**Minor**: **N.+**|(% style="width:1044px" %)(((

94

Data exchanged/disseminated using the old version of the CS can still be exchanged/disseminated using the new CS

95

)))

96

|(% style="width:874px" %)**Removal of one or more existing concepts**|(% style="width:154px" %)**Major: +.0**|(% style="width:1044px" %)Data exchanged/disseminated using the old version of the CS can no longer be exchanged/disseminated using the new version with less concepts

97

98

|(% colspan="3" %)**DATA STRUCTURE DEFINITION (DSD)**

99

|(% style="width:868px" %)**Type of change**|(% style="width:159px" %)**Impact**|(% style="width:1045px" %)**Comments**

100

|(% style="width:868px" %)**Addition of a dimension**|(% style="width:159px" %)**Major**: **+.0**|(% style="width:1045px" %)Adding a new dimension has a strong impact because a dimension represents the identifier of a dataset, thus requiring a remodelling of the data as existing structural validation will fail

101

|(% style="width:868px" %)**Addition of a mandatory attribute**|(% style="width:159px" %)**Major**: **+.0**|(% style="width:1045px" %)If the attribute is mandatory, the situation is the same as under point “Addition of a dimension”

102

|(% style="width:868px" %)**Addition of a conditional attribute**|(% style="width:159px" %)**Minor**: **N.+**|(% style="width:1045px" %)If the attribute is conditional backward compatibility is maintained

103

|(% style="width:868px" %)**Removal of a dimension or attribute**|(% style="width:159px" %)**Major**: **+.0**|(% style="width:1045px" %)Whatever the type of component, the change does not guarantee backward compatibility

104

105

For concrete examples, see the Appendix.

106

107

= 5. How versioning works for inter-dependent artefacts =

108

109

This section describes how version changes to inter-dependent or parent/child artefacts affect each other. For example, how a Concept Scheme is affected when one of the Code Lists that it references changes version.

110

111

Some artefacts have references to other artefacts. For example:

112

113

* each of a Concept Scheme’s Concepts may reference a Code List;

114

* a DSD can reference one or more Concept Schemes;

115

* each of a DSD’s Concepts may reference a Code List. (Note that if a Concept-Code List reference exists both in a DSD and a Concept Scheme, the Concept-Code List reference in the DSD overrides the reference in the Concept Scheme);

116

* a Hierarchical Code List references one or more Code Lists whose codes are arranged in the hierarchical structure.

117

118

In the text below, the following concepts will be used:

119

120

* **Parent artefact**: an artefact that contains a reference to another artefact. For example, a Concept Scheme is a parent to a Code List that it references, and the Code List is the child;

121

* **Child artefact**: an artefact that is referenced by another artefact. For example, a Code List is a child of a Concept Scheme that contains a reference to it, and the Concept Scheme is the parent.

122

123

It is important to note that a new version of a child artefact does not automatically trigger a version update of the parent artefact. A version change to the parent artefact is made only if the new version of the child artefact is adopted by the parent artefact.

124

125

==== a. Impact on parent artefact when child artefact version changes ====

126

127

The replacement of a reference with a different reference has the same impact for every artefact.

128

129

|(% colspan="3" %)**ALL ARTEFACTS**

130

|**Type of change**|**Impact**|**Comments**

131

|(((

132

**Replacement of a child artefact having a different version, but same id and**

**Agency**

)))|(((

**The child artefact version change is replicated in the**

**parent artefact**

)))|(((

If a child artefact (e.g. a Code List) has a minor version change, then the parent artefact (e.g. a Concept Scheme) should also have a minor version change.

141

142

If there are several child artefact version changes, the most severe impact is replicated in the parent artefact. For example, if two Code Lists have minor changes, and one Code List has a major change at the same time, the parent Concept Scheme has a major version change

143

)))

144

|(((

145

**Replacement of a referenced child artefact having a**

146

147

**different id or Agency**

148

)))|**The parent artefact version impact depends on the backward/ forward compatibility as shown in the tables above**|Technically, the child artefact is not considered to be related to the previous child artefact. It needs to be checked whether exchange contracts can still be guaranteed (backward/forward compatibility principle)

149

150

==== b. Addition or removal of referenced artefacts ====

151

152

| |(% colspan="2" %)**CONCEPT SCHEME (CS)**

153

|**Type of change**|**Impact**|**Comments**

154

|(((

155

**Addition or removal of a child**

156

157

**Code List**

158

)))|**Minor: N.+**|The child Code Lists in a Data Structure Definition have priority over those referenced in a Concept Scheme. Child Code Lists added to or removed from a Concept Scheme do not have a direct impact on the data exchange. Backward/forward compatibility depends on the way Code Lists are referenced in Data Structure Definitions referencing the concept scheme. This needs to be taken into account when creating a new version of a DSD accordingly

| |(% colspan="2" %)**DATA STRUCTURE DEFINITION (DSD)**

163

|**Type of change**|**Impact**|**Comments**

164

|**Addition or removal of a child Code List**|(((

165

**If same id and Agency, then the child artefact version change is replicated in the parent artefact.**

166

167

**If different id or Agency, impact wil depend on the backward/forward compatibility as shown in the tables above**

168

)))|(((

169

If a child Code List has a minor version change, then the DSD should also have a minor version change.

170

171

If there are several Code List version changes, the most severe impact is replicated in the DSD. For example, if two Code Lists have minor changes, and one Code List has a major change at the same time, the parent DSD has a major version change

172

)))

173

174

=== 6. Appendix - Examples ===

175

176

**Example 1 – Change to a Code List name, for clarification purposes**. **Patch Impact: N.M.+**

177

178

|**Id**|**Old Name**|**New Name**

179

|**CL_ADJUSTMENT**|Adjustment codes|Adjustment code list

180

181

**Example 2** – **Change to a Concept name, for clarification purposes. Patch impact: N.M.+**

182

183

|**Id**|**Old name**|**New name**

184

|**PRODUCT_TO**|Product classification|Product classification (input-output product*product)

185

186

**Example 3** – **Change in the substance of codes. Major impact: +.0**

187

188

|**Id**|**Old name**|**New name**

189

|**CP01115**|Other products|Pizza and quiche

190

191

**Example 4 **- **Aggregation, disaggregation or reorganisation of codes. Major impact**: **+.0**

192

193

|(% colspan="2" %)**AGGREGATION OF EXISTING CODES**

194

|**Old version**|**New version**

195

|**2011** Heifers (female bovine that never calved), live **2012** Cows, live|**2010** Heifers and cows, live

196

|(% colspan="2" %)Codes **2011** and **2012** are fully{{footnote}}i.e. without integration into or combination with another existing code.{{/footnote}} **removed** and replaced with one **brand new** code. In this case there is a many to 1 correspondence between the codes.

|(% colspan="2" %)**DISAGGREGATION OF EXISTING CODES**

201

|**Old version**|**New version**

202

|**1010** Live horses|(((

203

1. Pure bred breeding horses, live

204

1. Other horses, live

205

)))

206

|(% colspan="2" %)Code **1010** is fully **removed** and replaced with two **brand new** codes. In this case there is a 1 to m correspondence between the codes.

|(% colspan="2" %)**REORGANISATION OF EXISTING CODES**

211

|**Old version**|**New version**

212

|(((

213

**3010** Fowls, weighing ≤ 185 g

214

215

**3020** Ducks, , weighing ≤ 185 g

216

217

**3030** Other poultry, weighing ≤ 185 g

218

219

**3040** Fowls, weighing > 185 g

220

221

**3050** Ducks, , weighing > 185 g

222

223

**3060** Other poultry, weighing > 185 g

224

)))|**3025** Poultry, weighing ≤ 175 g **3045** Poultry, weighing > 175 g

225

|(% colspan="2" %)Codes **3010**, **3020**, **3030**, **3040**, **3050** and **3060** are fully removed and replaced with two brand new codes; furthermore the criterion for the classification used in the old version has been changed in the new version (185 g criterion versus 175 g criterion), so that it is not possible to exactly aggregate the codes from the old version to the codes of the new version (e.g. a part of **3010** goes to **3025**, another part to **3045**). In this case there is a m to n correspondence between the two sets of codes

226

227

**Example 5 – Changes to hierarchies in a Code List. Major impact: +.0**

228

229

|(% colspan="2" %)**ADDING A NEW CODE IN AN EXISTING HIERARCHY – CODE LIST**

230

|**Old version**|**New version**

231

|• 0213 - Beer o02131 - Lager beer o02132 - Other alcoholic beer|• 0213 - Beer o02131 - Lager beer o 02132 - Other alcoholic beer o **02133 - Low and non-alcoholic beer**

232

|(% colspan="2" %)Code 02133 has been added to hierarchy 0213

233

234

**Example 6 – Changes to hierarchies in a Hierarchical Code List. Major impact: +.0**

235

236

|(% colspan="2" %)**ADDING A NEW CODE IN AN EXISTING HIERARCHY – HIERARCHICAL CODE LIST**

237

|**Old version**|**New version**

238

|(((

239

• A1 - World (codelist ref. ECB@CL_AREAS@1.0) o E1 - Europe (ECB@CL_COUNTRIES@1.0)

240

241

ES - Spain FR - France

GR - Greece

IT - Italy o E4 - Africaetc.

246

)))|(((

247

• A1=World (codelist ref. ECB@CL_AREAS@1.0) o E1 =Europe (ECB@CL_COUNTRIES@1.0)

248

249

ES = Spain FR = FranceGR = Greece

IT = Italy

**DE= Germany**

o E4 =Africaetc.

)))

|(% colspan="2" %)The id of the hierarchical codes are assumed to be equal to those of the code lists referenced. The code DE has been added to hierarchy E1

258

259

**Example 7.1 – Dependencies between artefacts: Concept Scheme and Code List. Minor impact: N.+**

**New**

**version**

)))

|**CS_TRADE:Concept Scheme: References CL_OBS_STATUS v1.0 above**|(((

268

Adoption of new code X

269

270

**//Change type//**: Replacement of a child artefact having a different version, but the same id and

Agency

)))|(((

**Minor: N.+**

The child version impact is replicated in the parent

artefact

)))|**2.0**|**2.1**

**Example 7.2 – Dependencies between artefacts: Concept Scheme and Code List. Major impact: +.0**

**New**

**version**

)))

|(((

**CS_TRADE:Concept**

**Scheme:References**

293

294

**CL_OBS_STATUS v1.0 above**

295

)))|(((

296

Adoption of new CL_OBS_STATUS without U.

297

298

**//Change type//**: Replacement of a child artefact having a different version, but the same id and

Agency

)))|(((

**Major: +.0**

The child version impact is replicated in the parent artefact.

305

)))|**2.0**|**3.0**

306

307

**Example 7.3 – Dependencies between artefacts: Concept Scheme and Code List. Variable impact (see below)**

**New**

**version**

)))

|(% rowspan="3" %)**CL_XYZ: Code List**|(((

315

a) Maintenance agency changes from A to B for governance reasons.

316

317

Nothing else changes in the code list.

318

)))|(% rowspan="3" %)**New artefact**|(% rowspan="3" %)(((

**CL_XYZ**

**(Agency A)**

)))|(% rowspan="3" %)(((

**CL_XYZ**

**(Agency B)**

**(new maintenance agency)**

328

)))

329

|b) Maintenance agency changes from A to B and at the same time new codes are added

330

|c) Maintenance agency changes from A to B. Since B has different coding rules, the code list itself changes as well.

331

|(% rowspan="3" %)(((

332

**CS_TRADE: Concept Scheme: References**

333

334

**CL_XYZ (Agency A) **

335

)))|(% rowspan="3" %)(((

336

Replacement of a child artefact having a different Agency.

337

338

CL_XYZ (Agency A) changes to CL_XYZ (Agency B).

339

)))|(((

340

**Case a): Patch: N.M.+**

341

342

There is no impact on data exchange

343

)))|**2.0**|**2.0.1**

344

|(((

345

**Case b): Minor: N. +**

346

347

The impact is the same as a new minor version of the code list

348

)))|**2.0**|**2.1**

349

|(((

350

**Case c) Major: +.0**

351

352

The impact is the same as a new major version of the code list.

353

)))|**2.0**|**3.0**

354

355

**Example 7.4 – Dependencies between artefacts: Concept Scheme and DSD. Variable impact (see below)**

|(((

**TRADE: Data Structure Definition:**

361

362

**references Concepts C1 and C2**

)))|(((

None

Concept C3 is not used

367

)))|**None**|**1.0**|**1.0**

368

|** **| |** **|** **|** **

369

370

|(((

371

**TRADE: Data Structure Definition:**

372

373

**references Concepts C1 and C2**

)))|(((

None

Concept C3 is not used

378

)))|**None**|**1.0**|**1.0**

|(% colspan="5" %)

|(((

**TRADE: Data Structure Definition:**

383

384

**references Concepts C1 and C2**

385

)))|None concept C3 is not used|**None**|**1.0**|**1.0**

386

|(% colspan="5" %)**Remark: **Once a new version of the DSD is needed for some other reasons (e.g. a change in a code list), it is recommended to update all concept references to the newest available concept scheme if possible: i.e. DSD version 1.1 would then update its concept scheme references from 1.4 to 2.0.

|(% colspan="5" %)

|(((

**TRADE: Data Structure Definition:**

391

392

**references Concepts C1 and C2**

393

)))|Correction should be taken into account, concept C2 is used|(((

**Patch: N.M.+**

**//or//**

**None**

)))|**1.0**|(((

**1.0.1 //or//**

**1.0**

)))

|(% colspan="5" %)**Remark: **Since the change of a typo in a Concept of the Concept Scheme does not have a direct impact on the DSD itself (the link is by reference), there is strictly speaking no need to update the DSD. Both DSDs (1.0 and 1.0.1) will have exactly the same syntax. However, if maintainers want to highlight the correction for users of the DSD or for some other reason the DSD is updated anyway; it should reference the newer Concept Scheme.

**~ **

----

Wiki source code of Guidelines on the Versioning of SDMX Artefacts