7 Metadata Repository and Linking Data to Metadata
- Contents
7.1 Scope of this Chapter
The storage, retrieval and linkage to data of Reference Metadata (often known as “footnote” metadata) is becoming more and more important in the dissemination of statistics. All organizations that compile statistical data have reference metadata and it is often useful to data consumers for the data providers to disseminate this with the data. This can be done in a simple way by disseminating all such metadata regardless of whether it pertains to the data, or it can be done in a more intelligent way, and therefore more useful to the data consumer, by disseminating only that metadata which pertains to the data being viewed or downloaded.
This Chapter discusses the role of the metadata repository in the context of:
- its logical structure
- loading metadata
- retrieving metadata
7.2 Structure of the Metadata Repository
In SDMX all metadata is characterized as:
Examples are shown below. The “Metadata Attribute” is taken from the SDMX Cross Domain Concepts and the “Attached to” is a typical object to which it would be attached in a data dissemination scenario.
Metadata Attribute Description Attached To COMPILING_ORG The organisation compiling the data being reported Data Key – this can be a full or partial key. In thie specific example presented here it would probably be attached to the Reference Area dimension. COVERAGE_SECTOR Main economic or other sectors covered by the statistics CODE SOURCE_TYPE Characteristics and components of the raw statistical data used for compiling statistical aggregates DATAFLOW UNIT_MEASURE SERIES KEY COMMENT Supplementary descriptive text which can be attached to data or metadata CONCEPT ( if applicable to all uses of the Concept) or DIMENSION (if applicable to the use of a Concept in a specific DSD) OBS_STATUS Information on the quality of a value or an unusual or missing value OBSERVATION Therefore the conceptual structure of a Metadata Repository is:
Figure 14: Conceptual Structure of a Metadata Repository
This is only a conceptual structure and is purposefully over-simplistic as it is drawn to show conceptually how such a metadata repository may work.
The Metadata Key is comprised of one or more Object Type/Object Id. For instance, for a Code or a Concept it would be the Id of the Code or Concept such as the URN. For a specific key or partial key there would be two Objects, the DSD and the Data Key. Clearly, this is a conceptual structure as it is probable that the Metadata Key would combine the value of all of the individual Object Ids, as this would be necessary in order to find metadata quickly.
The Metadata Attribute Table may, of course, contain additional information depending on the actual requirements. For instance, it may not be necessary to reference the Metadata Structure Definition (MSD) providing the Concept name and Description are contained in the table. Also, it may be useful to add a “Short Name” to be used by applications such as web sites. Note that the metadata attribute structure can be hierarchical (i.e. a Metadata Attribute can have child Metadata Attributes). As each Metadata Attribute can have only one parent this structure is achieved by the Parent Attribute Id.
The way such
a repository would support the linking of data to metadata is shown below.
7.3 Linking Data to Metadata
7.3.1 Examples
7.3.2 Metadata Structure Definition
7.3.2.1 MSD Visualised in a Tool
Figure 15: MSD - Metadata Targets
Notes
The DATA_KEY_TARGET comprises two components:
- A Key Descriptor (dimension descriptor) – this specifies that a key (one or more dimension values) must be specified in the Metadata Set
- An Identifiable Object which must be a Dataflow – this specifies that a Dataflow must be specified in the Metadata Set. If required, the valid list of Dataflows can be specified in an Item Scheme such as a Category Scheme.
The combination of these two components specifies a data key in the context of a specific Dataflow.
The DATAFLOW_TARGET comprises just one component:
1. An Identifiable Object which must be a Dataflow
Figure 16: MSD - Report Structure
Notes
- There is only one Report Structure. The same structure is used for both Metadata Targets (Dataflow and Data Key).
- It is possible to have many Report Structure in an MSD, each referencing a different Metadata Target.
7.3.2.2 MSD SDMX-ML
Figure 17: MSD - Metadata Targets
Figure 18: MSD - Metadata Report
Note that the reasons of brevity content of many of the Metadata Attributes has been collapsed e.g. each has
a Concept identity and Local Representation.
7.3.3 Metadata Set
