5 Data Structure Definition and Dataset

5.1 Introduction

The DataStructureDefiniton is the class name for a structure definition for data. Some organisations know this type of definition as a “Key Family” and so the two names are synonymous. The term Data Structure Definition (also referred to as DSD) is used in this specification.

Many of the constructs in this layer of the model inherit from the SDMX Base Layer. Therefore, it is necessary to study both the inheritance and the relationship diagrams to understand the functionality of individual packages. In simple sub models these are shown in the same diagram but are omitted from the more complex sub models for the sake of clarity. In these cases, the inheritance diagram below shows the full inheritance tree for the classes concerned with data structure definitions.

There are very few additional classes in this sub model other than those shown in the inheritance diagram below. In other words, the SDMX Base gives most of the structure of this sub model both in terms of associations and in terms of attributes. The relationship diagrams shown in this section show clearly when these associations are inherited from the SDMX Base (see the Appendix “A Short Guide to UML in the SDMX Information Model” to see the diagrammatic notation used to depict this).

The actual SDMX Base construct from which the concrete classes inherit depends upon the requirements of the class for:

Annotation – AnnotableArtefact
Identification – IdentifiableArtefact
Naming – NameableArtefact
Versioning – VersionableArtefact
Maintenance – MaintainableArtefact

5.2 Inheritance View

5.2.1 Class Diagram

Figure 27 Class inheritance in the Data Structure Definition and Data Set Packages

5.2.2 Explanation of the Diagram

5.2.2.1 Narrative

Those classes in the SDMX metamodel which require annotations inherit from AnnotableArtefact. These are:

IdentifiableArtefact
DataSet
Key (and therefore SeriesKey and GroupKey)
Observation

Those classes in the SDMX metamodel which require annotations and global identity are derived from IdentifiableArtefact. These are:

NameableArtefact
ComponentList
Component

Those classes in the SDMX metamodel which require annotations, global identity, multilingual name and multilingual description are derived from NameableArtefact. These are:

VersionableArtefact
Item

The classes in the SDMX metamodel which require annotations, global identity, multilingual name and multilingual description, and versioning are derived from VersionableArtefact. These are:

MaintainableArtefact

Abstract classes which represent information that is maintained by Maintenance Agencies all inherit from MaintainableArtefact, they also inherit all the features of a VersionableArtefact, and are:

StructureUsage
Structure
ItemScheme

All the above classes are abstract. The key to understanding the class diagrams presented in this section are the concrete classes that inherit from these abstract classes.

Those concrete classes in the SDMX Data Structure Definition and Dataset packages of the metamodel which require to be maintained by Agencies all inherit (via other abstract classes) from MaintainableArtefact, these are:

Dataflow
DataStructureDefinition

The component structures that are lists of lists, inherit directly from Structure. A Structure contains several lists of components. The concrete class that inherits from Structure is:

DataStructureDefinition

A DataStructureDefinition contains a list of dimensions, a list of measures and a list of attributes.

The concrete classes which inherit from ComponentList and are subcomponents of the DataStructureDefinition are:

DimensionDescriptor – content is Dimension and TimeDimension
DimensionGroupDescriptor – content is an association to Dimension, TimeDimension
MeasureDescriptor – content is Measure
AttributeDescriptor – content is DataAttribute and an association to MetadataAttribute

The classes that inherit from Component are:

Measure

DimensionComponent and thereby its sub classes of Dimension and TimeDimension
Attribute and thereby its sub classes of DataAttribute and MetadataAttribute

The concrete classes identified above are the majority of the classes required to define the metamodel for the DataStructureDefinition. The diagrams and explanations in the rest of this section show how these concrete classes are related in order to support the functionality required.

5.3 Data Structure Definition – Relationship View

5.3.1 Class Diagram

Figure 28 Relationship class diagram of the Data Structure Definition excluding representation

5.3.2 Explanation of the Diagrams

5.3.2.1 Narrative

A DataStructureDefinition defines the Dimensions, TimeDimension, DataAttributes, and Measures, and associated Representations, that comprise the valid structure of data and related attributes that are contained in a DataSet, which is defined by a Dataflow. In addition, a DataStructureDefinition may be related to one MetadataStructureDefinition, in order to use the latter’s MetadataAttributes, by relating them to other Components within the DSD, as explained below.

The Dataflow may also have additional metadata attached that define qualitative information and Constraints on the use of the DataStructureDefinition such as the subset of Codes used in a Dimension (this is covered later in this document – see sections “Constraints” and “Data Provisioning”). Each Dataflow has a maximum of one DataStructureDefinition specified which defines the structure of any DataSets to be reported/disseminated.

There are two types of dimensions each having a common association to Concept:

• Dimension
• TimeDimension

Note that DimensionComponent can be any or all its sub classes i.e., Dimension, TimeDimension.

The DimensionComponent, DataAttribute, MetadataAttribute and Measure link to the Concept that defines its name and semantic (/conceptIdentity association to Concept). The DataAttribute, Dimension (but not TimeDimension) and Measure can optionally have a +conceptRole association with a Concept that identifies its role in the DataStructureDefinition, or one of the standard pre-defined roles, i.e., those published in "GUIDELINES FOR SDMX CONCEPT ROLES" by the SDMX SWG. The use of these roles is to enable applications to process the data in a meaningful way (e.g., relating a dimension value to a mapping vector). It is expected, beyond the standard roles published by the SWG, that communities (such as the official statistics community) will harmonise such roles within their community so that data can be exchanged and shared in a meaningful way within that community.

The valid values for a DimensionComponent, Measure, DataAttribute or MetadataAttribute, when used in this DataStructureDefinition, are defined by the Representation. This Representation is taken from the Concept definition (coreRepresentation) unless it is overridden in this DataStructureDefinition (localRepresentation) – see Figure 28. Note also that TimeDimension is constrained to specific FacetValueTypes. Moreover, the Representations of MetadataAttributes are specified in the corresponding MetadataStructureDefinition, linked by the DataStructureDefinition.

There will always be a DimensionDescriptor grouping that identifies all of the Dimension comprising the full key. Together the Dimensions specify the key of an Observation.

The DimensionComponent can optionally be grouped by multiple GroupDimensionDescriptors each of which identifies the group of Dimensions that can form a partial key. The GroupDimensionDescriptor must be identified (GroupDimensionDescriptor.id) and this is used in the GroupKey of the DataSet to declare which DataAttributes or MetadataAttributes are reported at this group level in the DataSet.

There can be a maximum of one TimeDimension specified in the DimensionDescriptor. The TimeDimension is used to specify the Concept used to convey the time period of the observation in a data set. The TimeDimension must contain a valid representation of time and cannot be coded.

here can be one or more Measures under the MeasureDescriptor. Measures represent the observable phenomena. Each Measure may have a valid representation, a maxOccurs attribute limiting the maximum number of values per Measure (which may be set to 'unbounded' for unlimited occurrences), as well as a minOccurs attribute, indicating the minimum required number of values, when the Measure is reported. If minOccurs or maxOccurs are omitted (they both default to ‘1’), the Measure is considered to take a single value; otherwise, it is an array. Moreover, the usage attribute indicates whether a Measure must be reported or not, by the corresponding values: mandatory or optional.

The AttributeDescriptor may contain one or more Attributes, i.e., at least one DataAttribute definition or one MetadataAttribute reference.

The DataAttribute defines a characteristic of data that are collected or disseminated and is grouped in the DataStructureDefinition by a single AttributeDescriptor. The DataAttribute can be indicated if it must be reported or not, by the corresponding value of the usage attribute: i.e., mandatory or optional. The property minOccurs specifies the minimum number of array values to be included when the DataAttribute is reported. Moreover, a maxOccurs attribute indicates whether the DataAttribute may need to report more than one values, i.e., an array of values. The DataAttribute may play a specific role in the structure and this is specified by the +role association to the Concept that identifies its role.

The MetadataAttribute defines reference metadata that may be collected or disseminated and is grouped together with DataAttribute under the AttributeDescriptor.

A DataAttribute or a MetadataAttribute (i.e., an AttributeComponent) is specified as being +relatedTo an AttributeRelationship, which defines the constructs to which the AttributeComponent is to be reported within a DataSet. An AttributeComponent can be specified as being related to one of the following artefacts:

• All data within the dataset (DataflowRelationship) – this is equivalent to attaching an Attribute to all data within the Dataflow.
• Dimension or set of Dimensions (DimensionRelationship)
• Set of Dimensions specified by a GroupKey (GroupRelationship – this is retained for compatibility reasons – or +groupKey of the DimensionRelationship)
• Observation (ObservationRelationship)
• In addition to the positioning of an AttributeComponent within a DataSet, another relationship indicates the Measure(s) for which the AttributeComponent is reported. Regardless of the position of the AttributeComponent within the DataSet, the AttributeComponent may concern one, more than one, or all Measures included in the DSD. This is expressed using the MeasureRelationship class, which relates a DataAttribute to one or more Measures. Lack of the MeasureRelationship defaults to a relationship to all Measures.

Figure 29: Attribute Attachment Defined in the Data Structure Definition

The following table details the possible relationships a DataAttribute may specify. Note that these relationships are mutually exclusive, and therefore only one of the following is possible.

Figure 30: Representation of DSD Components

Each of Dimension, TimeDimension, Measure, DataAttribute and MetadataAttribute can have a Representation specified (using the localRepresentation association). If this is not specified in the DataStructureDefinition then the representation specified for Concept (coreRepresentation) is used. Measure, and DataAttribute may be also represented by multilingual text (as seen in the DataSet diagram further down). An exception is the MetadataAttribute, where its Representation is specified in the MetadataStructureDefinition.

A DataStructureDefinition can be extended to form a derived DataStructureDefinition. This is supported in the StructureMap.

5.3.2.2 Definitions

The explanation of the classes, attributes, and associations comprising the Representation is described in the section on the SDMX Base.

5.4 Data Set – Relationship View

5.4.1 Context

A data set comprises the collection of data values and associated metadata that are collected or disseminated according to a known DataStructureDefinition.

5.4.2 Class Diagram

Figure 31: Class Diagram of the Data Set

5.4.3 Explanation of the Diagram

5.4.3.1 Narrative – Data Set

Note that the DataSet must conform to the DataStructureDefinition associated to the Dataflow for which this DataSet is an “instance of data”. Whilst the model shows the association to the classes of the DataStructureDefinition, this is for conceptual purposes to show the link to the DataStructureDefinition. In the actual DataSet as exchanged there must, of course, be a reference to the DataStructureDefinition and optionally a Dataflow or a ProvisionAgreement, but the DataStructureDefinition is not necessarily exchanged with the data. Therefore, the DataStructureDefinition classes are shown in the grey areas, as these are not a part of the DataSet when the DataSet is exchanged. However, the structural metadata in the DataStructureDefinition can be used by an application to validate the contents of the DataSet in terms of the valid content of a KeyValue as defined by the Representation in the DataStructureDefinition.

An organisation playing the role of DataProvider can be responsible for one or more DataSet.

A DataSet is formatted as a DataStructureDefinition specific data set (StructureSpecificDataSet). The structured data set is structured according to one specific DataStructureDefinition; hence the latter is required for validation at the syntax level.

A DataSet is a collection of a set of Observations that share the same dimensionality, which is specified by a set of unique components (Dimension, TimeDimension) defined in the DimensionDescriptor of the DataStructureDefinition, together with associated

AttributeValues that define specific characteristics about the artefact to which it is attached – Observations, set of Dimensions. It can be structured in terms of a SeriesKey to which Observations are reported.

The Observation can be the value(s) of the variable(s) being measured for the Concept associated to the Measure(s) in the MeasureDescriptor of the DataStructureDefinition. Each Observation associates one or more ObservationValues with a KeyValue (+observationDimension) which is the value for the “Dimension at the Observation Level”. Any Dimension can be specified as being the “Dimension at the Observation Level”, and this specification is made at the level of the DataSet (i.e., it must be the same Dimension for the entire DataSet).

The KeyValue is a value for one of TimeDimension or Dimension specified in the DataStructureDefinition. If it is a Dimension, it can be coded (CodedKeyValue) or uncoded (UncodedKeyValue). If it is the TimeDimension then it is a TimeKeyValue. The actual value that the CodedDimensionValue can take must be one of the Codes in the Codelist specified as the Representation of the Dimension in the DataStructureDefinition.

An ObservationValue can be coded – this is the CodedObservation – or it can be uncoded – this is the UncodedObservation. In the case of uncoded observations, the values may be multilingual – expressed via the TextMeasureValue – or not (OtherUncodedMeasureValue).

The GroupKey is a subunit of the Key that has the same dimensionality as the SeriesKey but defines a subset of the KeyValues of the SeriesKey. Its sub dimension structure is defined in the GroupDimensionDescriptor of the DataStructureDefinition identified by the same id as the GroupKey. The id identifies a “type” of group and the purpose of the GroupKey is to report one or more AttributeValue that are contained at this group level. The GroupKey is present when the GroupDimensionDescriptor is related to the GroupRelationship in the DataStructureDefinition. There can be many types of groups in a DataSet. If the Group is related to the DimensionRelationship in the DataStructureDefinition then the AttributeValue will be reported with the appropriate dimension in the SeriesKey or Observation.

In this way each of SeriesKey, GroupKey, and Observation can have zero or more AttributeValues that define some metadata about the object to which it is associated. The AttributeValue may be either a DataAttributeValue or a MetadataAttributeValue, representing values of DataAttributes defined in the DSD or MetadataAttributes of the linked MSD, respectively. The allowable Concepts and the objects to which these metadata can be associated (attached) are defined in the DataStructureDefinition and the linked MetadataStructureDefinition.

The AttributeValue links to the object type (SeriesKey, GroupKey, Observation) to which it is associated.

5.4.3.2 Definitions