Skip to Content

Wiki source code of 6 Codelist

Version 14.5 by Helena on 2025/05/19 14:23
Show last authors
1 {{box title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 As of [[SDMX>>doc:xwiki:Glossary.Statistical data and metadata exchange.WebHome]] 3.0, Codelists have gained new features like geospatial properties, inheritance and extension. Moreover, [[hierarchies>>doc:xwiki:Glossary.Hierarchy.WebHome]] (used to build complex [[hierarchies>>doc:xwiki:Glossary.Hierarchy.WebHome]] of one or more Codelists) are now linked to other [[Artefacts>>doc:xwiki:Glossary.Artefact.WebHome]] in order to facilitate the formers' usage in dissemination or other scenarios. For all geospatial related features, as well as the new Geographical Codelist, please refer to section 7.
6
7 == 6.1 Codelist extension and discriminated unions ==
8
9 A Codelist can extend one or more Codelists. Codelist extensions are defined as a list of references to parent Codelists. The order of the references is important when it comes to conflict resolution on [[Code>>doc:xwiki:Glossary.Code.WebHome]] Ids. When two Codelists have the same [[Code>>doc:xwiki:Glossary.Code.WebHome]] Id, the Codelist referenced later takes priority. In the example below, the [[code>>doc:xwiki:Glossary.Code.WebHome]] 'A', exists in both CL_INDICATOR and CL_SERIES. The Codelist CL_INDICATOR_EX will contain the [[code>>doc:xwiki:Glossary.Code.WebHome]] 'A' from CL_SERIES as this was the second Codelist to be referenced in the sequence of references.
10
11 [[image:1747384656299-761.png]]
12
13 (% class="wikigeneratedid" id="HFigure7:Codelistextension" %)
14 **Figure 7: Codelist extension**
15
16 As the extended Codelist, CL_INDICATOR_EX in this example, may also define its own [[Codes>>doc:xwiki:Glossary.Code.WebHome]], these take the ultimate priority over any referenced Codelists. If CL_INDICATOR_EX defines [[Code>>doc:xwiki:Glossary.Code.WebHome]] 'A', then this will be used instead of [[Code>>doc:xwiki:Glossary.Code.WebHome]] 'A' from CL_INDICATOR and CL_SERIES, as shown below:
17
18 [[image:1747384672686-615.png]]
19
20 (% class="wikigeneratedid" id="HFigure8:CodelistextensionwithnewCodes" %)
21 **Figure 8: Codelist extension with new Codes**
22
23 === 6.1.1 Prefixing Code Ids ===
24
25 A reference to a Codelist may contain a prefix. If a prefix is provided, this prefix will be applied to all the [[codes>>doc:xwiki:Glossary.Code.WebHome]] in the Codelist before they are imported into the extended Codelist. Following the above example if the CL_SERIES reference includes a prefix of 'SER_' then the resulting Codelist would contain 7 [[codes>>doc:xwiki:Glossary.Code.WebHome]], A, E, F, X, Y, SER_A, SER_B, SER_C.
26
27 [[image:1747384699130-401.png]]
28
29 (% class="wikigeneratedid" id="HFigure9:ExtendedCodelistwithprefix" %)
30 **Figure 9: Extended Codelist with prefix**
31
32 === 6.1.2 Including / Excluding Specific Codes ===
33
34 The default behaviour of extending another Codelist is to import all [[Codes>>doc:xwiki:Glossary.Code.WebHome]]. However, an explicit list of [[Code>>doc:xwiki:Glossary.Code.WebHome]] Ids may be provided for explicit inclusion or exclusion. This list of Ids may contain wildcards using the same notation as (% style="color:#2ecc71" %)Constraints(%%) ~(%). Cascading values is also supported using the same syntax as the (% style="color:#2ecc71" %)Constraints(%%). The list of Ids is either a list of excluded items, or included items, exclusion and inclusion is not supported against a single Codelist.
35
36 [[image:1747384712935-862.png]]
37
38 (% class="wikigeneratedid" id="HFigure10:ExtendedCodelistwithinclude2Fexcludeterms" %)
39 **Figure 10: Extended Codelist with include/exclude terms**
40
41 === 6.1.3 Parent Ids ===
42
43 Parent Ids are preserved in the extended Codelist if they can be. If a [[Code>>doc:xwiki:Glossary.Code.WebHome]] is inherited but its parent [[Code>>doc:xwiki:Glossary.Code.WebHome]] is excluded, then the [[Code>>doc:xwiki:Glossary.Code.WebHome]]'s parent Id will be removed. This rule is also true if the parent [[Code>>doc:xwiki:Glossary.Code.WebHome]] is excluded because it is overwritten by another [[Code>>doc:xwiki:Glossary.Code.WebHome]] with the same Id from another Codelist. This ensures the parent Ids do not inadvertently link to [[Codes>>doc:xwiki:Glossary.Code.WebHome]] originating from different Codelists, and also prevents circular references from occurring.
44
45 (% class="wikigeneratedid" %)
46 [[image:1747384741039-595.png]]
47
48 (% class="wikigeneratedid" id="HFigure11:ParentCodeincluded" %)
49 **Figure 11: Parent Code included**
50
51 [[image:1747384753771-913.png]]
52
53 **Figure 12: Parent Code from different extended Codelist**
54
55 [[image:1747384769943-748.png]]
56
57 **Figure 13: Parent Code overridden by local Code**
58
59 [[image:1747384788183-252.png]]
60
61 **Figure 14: Parent Code not included**
62
63 === 6.1.4 Discriminated Unions ===
64
65 A common use case solved in [[SDMX>>doc:xwiki:Glossary.Statistical data and metadata exchange.WebHome]] 3.0 is that of discriminated unions, i.e., dealing with classification or breakdown "variants" which are all valid but mutually exclusive. For example, there are many (% style="color:#2ecc71" %)versions(%%) of the international classification for [[economic activities>>doc:xwiki:Glossary.Economic activity.WebHome]] "ISIC". In [[SDMX>>doc:xwiki:Glossary.Statistical data and metadata exchange.WebHome]], classifications are enumerated (% style="color:#2ecc71" %)concepts(%%), normally modelled as [[dimensions>>doc:xwiki:Glossary.Dimension.WebHome]] corresponding to breakdowns. Each enumerated (% style="color:#2ecc71" %)concept(%%) is associated to one and only one [[code>>doc:xwiki:Glossary.Code.WebHome]] list.
66
67 To support this use case, the following have to be considered:
68
69 * **Independent Codelists per variant**: Having each variant in a separate Codelist facilitates the maintenance and allows keeping the original [[codes>>doc:xwiki:Glossary.Code.WebHome]], even if different (% style="color:#2ecc71" %)versions(%%) of the classification have the same [[code>>doc:xwiki:Glossary.Code.WebHome]] for different (% style="color:#2ecc71" %)concepts(%%). For example, in ISIC Rev. 4 the [[code>>doc:xwiki:Glossary.Code.WebHome]] "A" represents "Agriculture, forestry and fishing", while in ISIC 3.1 "A" means "Agriculture, hunting and forestry".
70 * **Prefixing [[Code>>doc:xwiki:Glossary.Code.WebHome]] Ids**: When extending Codelists, the reference to an extension Codelist may contain a prefix. If a prefix is provided, this prefix will be applied to all the [[codes>>doc:xwiki:Glossary.Code.WebHome]] in the Codelist before they are imported into the extended Codelist. In this case, the reference to CL_ISIC4 includes a prefix of "ISIC4_" and the reference to ISIC3 includes "ISIC3_", so the resulting Codelist will have no conflict for the "A" items which will become "ISIC3_A" and "ISIC4_A".
71 * **Including / Excluding Specific [[Codes>>doc:xwiki:Glossary.Code.WebHome]]**: As explained above, there will be independent DFs/PAs with specific (% style="color:#2ecc71" %)Constraint(%%) attached, in order to keep the proper items according to the variant in use by each data provider.
72
73 For example, assuming:
74
75 * [[DSD>>doc:xwiki:Glossary.Data structure definition.WebHome]] DSD_EXDU contains a [[Dimension>>doc:xwiki:Glossary.Dimension.WebHome]]: ACTIVITY enumerated by CL_ACTIVITY.
76 * CL_ACTIVITY has no items and is extended by:
77 * CL_ISIC4, prefix="ISIC4_"
78 * CL_ISIC3, prefix="ISIC3_"
79 * CL_NACE2, prefix="NACE2_"
80 * CL_AGGR, prefix="AGGR_"
81 * [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]] DF1, with a DataConstraint CC_NACE2, CubeRegion for ACTIVITY and Value="NACE2_%"
82 * [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]] DF2, with a DataConstraint CC_ISIC3, CubeRegion for ACTIVITY and Value="ISIC3_%"
83 * [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]] DF3, with a DataConstraint CC_ISIC4, CubeRegion for ACTIVITY and Value="ISIC4_%", Value="AGGR_TOTAL", Value="AGGR_Z"
84
85 The discriminated unions are achieved, by requesting any of the above [[Dataflows>>doc:xwiki:Glossary.Dataflow.WebHome]] with references="all" and detail="referencepartial": returns CL_ACTIVITY with the corresponding extensions resolved and the DataConstraint, referencing the [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]], applied. Thus, the CL_ACTIVITY will only include [[Codes>>doc:xwiki:Glossary.Code.WebHome]] prefixed according to the [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]], i.e.:
86
87 * Prefix "NACE2_%" for DF1;
88 * Prefix "ISIC3_%" fo DF2;
89 * Prefix "ISIC4_%" for DF3; note that [[Codes>>doc:xwiki:Glossary.Code.WebHome]] "AGGR_TOTAL" and "AGGR_Z" are also included in this case.
90
91 == 6.2 Linking Hierarchies ==
92
93 To facilitate the usage of [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] within other [[SDMX>>doc:xwiki:Glossary.Statistical data and metadata exchange.WebHome]] [[Artefacts>>doc:xwiki:Glossary.Artefact.WebHome]], the HierarchyAssociation is defined to link any [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] with any IdentifiableArtefact within a specific context.
94
95 The HierarchyAssociation is a simple [[Artefact>>doc:xwiki:Glossary.Artefact.WebHome]] operating like a Categorisation. The former specifies three references:
96
97 * The link to a [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]];
98 * The link to the IdentifiableArtefact that the [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] is linked (e.g., a [[Dimension>>doc:xwiki:Glossary.Dimension.WebHome]]);
99 * The link to the context that the linking is taking place (e.g., a [[DSD>>doc:xwiki:Glossary.Data structure definition.WebHome]]).
100
101 As an example, let’s assume:
102
103 * A [[DSD>>doc:xwiki:Glossary.Data structure definition.WebHome]] with a **COUNTRY** [[Dimension>>doc:xwiki:Glossary.Dimension.WebHome]] that uses Codelist **CL_AREA** as representation.
104 * A [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] (e.g., **EU_COUNTRIES**) that builds a [[hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] for the **CL_AREA** Codelist.
105 * In order to use this [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] for data of a [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]] (e.g., **EU_INDICATORS**), we need to build the following HierarchyAssociation:
106 * Links to the [[Hierarchy>>doc:xwiki:Glossary.Hierarchy.WebHome]] **EU_COUNTRIES (what is associated?)**
107 * Links to the [[Dimension>>doc:xwiki:Glossary.Dimension.WebHome]] **COUNTRY (where is it associated?)**
108 * Links to the context: [[Dataflow>>doc:xwiki:Glossary.Dataflow.WebHome]] **EU_INDICATORS (when is it associated?)**
109 * The above are also shown in the schematic below:
110
111 [[image:1747384831345-304.png]]
112
113 **Figure 15: Hierarchy Association**