Wiki source code of 4 Data and Metadata Creation and Reporting
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{box title="**Contents**"}} | ||
| 2 | {{toc/}} | ||
| 3 | {{/box}} | ||
| 4 | |||
| 5 | == 4.1 Scope of this Chapter == | ||
| 6 | |||
| 7 | [[image:SDMX_2-1_User_Guide_draft_0-1_html_ac1357efb5b550e1.jpg||data-xwiki-image-style-alignment="center" height="375" width="295"]] | ||
| 8 | |||
| 9 | This chapter covers the creation of data and metadata for reporting purposes, starting with the definition of the structures which the data and metadata are to be reported against. This chapter references two sets of samples, the details of which are provided in Annex 3 – Data and Metadata Samples. This section focuses on the details of the fundamental features of [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] whereas the samples referenced in the Annex demonstrate the syntactical [[representation>>doc:sdmx:Glossary.Representation.WebHome]] of these features in [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]]. | ||
| 10 | |||
| 11 | == 4.2 Basics == | ||
| 12 | |||
| 13 | Fundamental to defining a structure definition is defining the [[concepts>>doc:Glossary.Concept.WebHome]] which describe the information contained in the data or metadata and the [[code lists>>doc:sdmx:Glossary.Code list.WebHome]] which provide specific values for these [[concepts>>doc:Glossary.Concept.WebHome]]. | ||
| 14 | |||
| 15 | === 4.2.1 Defining Concepts === | ||
| 16 | |||
| 17 | Any [[component>>doc:sdmx:Glossary.Component.WebHome]] of a data or [[metadata structure definition>>doc:sdmx:Glossary.Metadata structure definition.WebHome]] must take its semantic from a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). This concept is important in that it: | ||
| 18 | |||
| 19 | 1. Provides a detailed definition of the [[component>>doc:sdmx:Glossary.Component.WebHome]] which describes the structure of the data or metadata. | ||
| 20 | 1. Can allow data and metadata for different structures to be comparable when (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are reused. | ||
| 21 | |||
| 22 | It is important when defining (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) to first consider (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) which are already defined within a given community, whether it be the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] community as a whole or a smaller community of users in a particular sector. As general rule, one should not define a new (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) when an existing (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) will suffice. By reusing an existing (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), data and metadata are more easily understood in a wider range of applications. This leads to greater interoperability and [[comparability>>doc:sdmx:Glossary.Comparability.WebHome]] of data/metadata. | ||
| 23 | |||
| 24 | Assuming new (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are to be defined, the first step is to determine an appropriate grouping for (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%). In [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], all (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are defined in schemes. These schemes serve to group similar (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) into groups which can be useful for maintenance purposes. One possible distinction to be used to determine the grouping of (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) is their intended usage. If (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are only to be used for metadata, it may be best to group these into a single scheme. Similarly, if (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are only to be used for data, these too may be grouped into a single scheme. It is important to consider that (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) themselves are not versioned, rather the schemes in which they are defined are versioned. This means that if any property of a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) is to change, the [[version>>doc:Glossary.Version.WebHome]] of the scheme in which it is defined must change if the scheme is marked as “final”. This in effect versions all (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) defined in that scheme. Therefore, when grouping (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) into schemes it is important to consider the stability of the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) definitions. | ||
| 25 | |||
| 26 | A (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) itself consists of three main [[components>>doc:sdmx:Glossary.Component.WebHome]]: | ||
| 27 | |||
| 28 | 1. Its identification, which must be unique within the scheme. | ||
| 29 | 1. Its name. | ||
| 30 | 1. Its description. | ||
| 31 | |||
| 32 | Note that the description is not mandatory, but is highly recommended in order to provide a more complete definition of a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). When you consider that these (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are the building blocks which are constructed to define the structure of all data and metadata in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]], it should be apparent why complete definitions are important. | ||
| 33 | |||
| 34 | In addition to the basic definition properties of the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) can reference another (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) from within the same scheme as its parent (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). The exact nature of this parent child relationship is not strictly enforced by the standard, but it is typically used to denote the child (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) is a specialization of the parent. For example, one may have a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) which defines a [[reference area>>doc:sdmx:Glossary.Reference area.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) as a "geographic area to which a measured statistical phenomenon relates". In order to allow for more specific types of [[references areas>>doc:sdmx:Glossary.Reference area.WebHome]], one might define (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) for countries as well as groups of geographically similar countries (e.g. continents) and political countries (e.g. military or economic alliances). These child (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) could reference the [[reference area>>doc:sdmx:Glossary.Reference area.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) as a parent in order to note that they are specializations of a [[reference area>>doc:sdmx:Glossary.Reference area.WebHome]]". An example of this can be seen in the Common Structures sample set. In this sample the [[confidentiality status>>doc:sdmx:Glossary.Confidentiality - status.WebHome]] references the [[confidentiality>>doc:sdmx:Glossary.Confidentiality.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) as a parent. The status is a specialization of a general [[confidentiality>>doc:sdmx:Glossary.Confidentiality.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). | ||
| 35 | |||
| 36 | < note the test for whether something is a specialization is to ask the question “is xxx a type of yyy” In the Cross Domain (% style="color:#e74c3c" %)[[Concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) the question would be “Is [[confidentiality status>>doc:sdmx:Glossary.Confidentiality - status.WebHome]] a type of [[confidentiality>>doc:sdmx:Glossary.Confidentiality.WebHome]]”. The answer is “no”. The problem here is that the cross domain (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) are grouped, often in a structural and not semantic sense. This, unfortunately, is carried over in the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] and needs to be rectified.> | ||
| 37 | |||
| 38 | When a [[component>>doc:sdmx:Glossary.Component.WebHome]] in a structure definition takes its semantic from a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), it always is provided a value in the data or metadata reported against the structure. A (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) can define the default nature of these values. This is done with a definition of a //core [[representation>>doc:sdmx:Glossary.Representation.WebHome]]//. The core [[representation>>doc:sdmx:Glossary.Representation.WebHome]] can either be a un-coded text format, in which a data type is defined along with [[facets>>doc:sdmx:Glossary.Facet.WebHome]] which serve to further restrict that data type, or a coded enumeration in which a [[code list>>doc:sdmx:Glossary.Code list.WebHome]] can be referenced. In the case where an enumerated core [[representation>>doc:sdmx:Glossary.Representation.WebHome]] is defined, the nature of the enumeration values can be described in the same manner that the un-coded text format could be. Note that if a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) is not provided a core [[representation>>doc:sdmx:Glossary.Representation.WebHome]], it is assumed to have an unbounded value set (i.e. it could be represented by a variety of [[code lists>>doc:sdmx:Glossary.Code list.WebHome]] or un-enumerated formats when used in a [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] or [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]). However, as with any core [[representation>>doc:sdmx:Glossary.Representation.WebHome]], a usage of the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) in a structure definition can always provide a local [[representation>>doc:sdmx:Glossary.Representation.WebHome]] which overrides the core [[representation>>doc:sdmx:Glossary.Representation.WebHome]] of the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). | ||
| 39 | |||
| 40 | Finally, in order to provide a more complete conceptual definition, the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) can reference an ISO 11179 (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). The intention of this reference is to reference the ISO 11179 (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) definition from which this [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) definition is derived. | ||
| 41 | |||
| 42 | === 4.2.2 Defining Code Lists === | ||
| 43 | |||
| 44 | Any [[component>>doc:sdmx:Glossary.Component.WebHome]] in a structure definition can specify an enumeration of possible values. These enumerations are defined as [[code lists>>doc:sdmx:Glossary.Code list.WebHome]]. A [[code list>>doc:sdmx:Glossary.Code list.WebHome]] contains a value set of enumerations and the names and descriptions of what is represented by the coded value. | ||
| 45 | |||
| 46 | The [[code list>>doc:sdmx:Glossary.Code list.WebHome]] itself is provided an identity, name, and description. The [[codes>>doc:sdmx:Glossary.Code.WebHome]] comprising the [[code list>>doc:sdmx:Glossary.Code list.WebHome]] have the same properties. Note that the identification of the [[code>>doc:sdmx:Glossary.Code.WebHome]] is its [[code>>doc:sdmx:Glossary.Code.WebHome]] value which will be used in any data/metadata sets. | ||
| 47 | |||
| 48 | [[Codes>>doc:sdmx:Glossary.Code.WebHome]] can also be arranged into simple [[hierarchies>>doc:sdmx:Glossary.Hierarchy.WebHome]], where any [[code>>doc:sdmx:Glossary.Code.WebHome]] can reference another [[code>>doc:sdmx:Glossary.Code.WebHome]] within the same list as its parent. Note that [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] does not formally define the exact nature of this relationship (e.g. whether the child [[code>>doc:sdmx:Glossary.Code.WebHome]] is additive to the parent and if so, whether there is a weight associated with the [[code>>doc:sdmx:Glossary.Code.WebHome]]). None the less, it is often useful to capture the fact that some formal relationship does exist, if only to allow for more detailed descriptions of these [[code lists>>doc:sdmx:Glossary.Code list.WebHome]] to be accessed in order to properly understand the data/metadata. | ||
| 49 | |||
| 50 | == 4.3 Data == | ||
| 51 | |||
| 52 | Fundamental to [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is the exchange of data, or more specifically multi-dimensional data against a known structure. Data in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] starts with the structure which defines it. | ||
| 53 | |||
| 54 | This section details the data structure and its relationship to [[data sets>>doc:sdmx:Glossary.Data set.WebHome]]. | ||
| 55 | |||
| 56 | === 4.3.1 Defining Data Structures === | ||
| 57 | |||
| 58 | A data structure is a collection of [[components>>doc:sdmx:Glossary.Component.WebHome]] which define what is being measured and what additional properties (metadata) can be transmitted alongside the actual observed data values. | ||
| 59 | |||
| 60 | ==== 4.3.1.1 Data Structure Components ==== | ||
| 61 | |||
| 62 | [[image:SDMX_2-1_User_Guide_draft_0-1_html_f1c1d2104fe28af3.jpg||data-xwiki-image-style-alignment="center" height="301" width="575"]] | ||
| 63 | |||
| 64 | (% style="text-align: center;" %) | ||
| 65 | **{{id name="image_3"/}}Figure 3: Schematic of the Data Structure Definition** | ||
| 66 | |||
| 67 | Each [[component>>doc:sdmx:Glossary.Component.WebHome]] must reference a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) from which it takes its semantic. This (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) defines the meaning of the [[component>>doc:sdmx:Glossary.Component.WebHome]]. Typically, the [[component>>doc:sdmx:Glossary.Component.WebHome]] will also take its identity from this (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%), although it is possible for a (data) structure-specific identity to be created. Regardless of whether the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) identity is used or a local identifier is assigned, the identifier of a [[component>>doc:sdmx:Glossary.Component.WebHome]] must be unique within the scope of all data structure [[components>>doc:sdmx:Glossary.Component.WebHome]]. | ||
| 68 | |||
| 69 | A [[component>>doc:sdmx:Glossary.Component.WebHome]] can also inherit the [[representation>>doc:sdmx:Glossary.Representation.WebHome]] of the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) from which it takes its semantic, or it can provide a data structure specific [[representation>>doc:sdmx:Glossary.Representation.WebHome]] for the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). This [[representation>>doc:sdmx:Glossary.Representation.WebHome]] can be either [[code>>doc:sdmx:Glossary.Code.WebHome]] or un-coded, although some [[component>>doc:sdmx:Glossary.Component.WebHome]] types have more restrictive representations. | ||
| 70 | |||
| 71 | A data structure [[component>>doc:sdmx:Glossary.Component.WebHome]] can also reference a collection of (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) for the purpose of identifying specific roles that the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) serves in the data structure. For example, a user community may define a collection of (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) which are intended to note special roles for data structure [[components>>doc:sdmx:Glossary.Component.WebHome]] (e.g. the [[unit of measure>>doc:sdmx:Glossary.Unit of measure.WebHome]]). A [[component>>doc:sdmx:Glossary.Component.WebHome]] can reference any of these (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) in order to specify that the role identified by the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) is being served by the [[component>>doc:sdmx:Glossary.Component.WebHome]]. Note that a [[component>>doc:sdmx:Glossary.Component.WebHome]] is implied to serve the role of the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) from which it takes its identity, therefore it does not need to reference this (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) again. | ||
| 72 | |||
| 73 | ===== //4.3.1.1.1 Primary Measure// ===== | ||
| 74 | |||
| 75 | Every data structure must define a primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]]. This [[component>>doc:sdmx:Glossary.Component.WebHome]] always has a fixed identifier (OBS_VALUE), and its purpose is to given a consistent data structure [[artefact>>doc:sdmx:Glossary.Artefact.WebHome]] where the observed value can be found. This makes the processing of data messages much more consistent, as the measured value is always readily found. The primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]] can define a [[representation>>doc:sdmx:Glossary.Representation.WebHome]], or it can inherit the [[representation>>doc:sdmx:Glossary.Representation.WebHome]] from the (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) from which it takes its semantic (this (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) can have, but need not have, and Id of OBS_VALUE). | ||
| 76 | |||
| 77 | ===== //4.3.1.1.2 Dimensions// ===== | ||
| 78 | |||
| 79 | The identification of the phenomenon being measured is achieved through the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] descriptor. Any given data structure must have at least one [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. The [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] (in some models these are known as “classificatory variables”) reference (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) which define identifying properties of the phenomenon being measured. In a [[dataset>>doc:sdmx:Glossary.Data set.WebHome]], each [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is given a value. This set of [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] values is often referred to as a key. In any given [[data set>>doc:sdmx:Glossary.Data set.WebHome]], there can only be one observed value and collection of data [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] (metadata) values per key. Therefore, a key uniquely identities any observed phenomenon. | ||
| 80 | |||
| 81 | There are two specialized [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] that can be defined only once for a data structure. The first is the time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. The time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] represents the point in time at which the phenomenon was observed. If the explicit time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is used, then it must use any or all of the allowable time [[representations>>doc:sdmx:Glossary.Representation.WebHome]] defined in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]. These [[representations>>doc:sdmx:Glossary.Representation.WebHome]] are: | ||
| 82 | |||
| 83 | 1. A Gregorian calendar period (which can be any or all of a year, a month and year, or a date) | ||
| 84 | 1. A standard reporting period (which can be any or all of a year, a semester, a trimester, a quarter, a week, or a day). Each reporting period exists in the context of a reporting year which is defined by a start day (expressed as a month and day). This start day is communicated in a specialized data [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] which will be discussed later in this section. | ||
| 85 | 1. A distinct duration, which consists of a start date and time and a duration | ||
| 86 | 1. A distinct point in time (i.e. a full time stamp) | ||
| 87 | |||
| 88 | Each of theses time [[representation>>doc:sdmx:Glossary.Representation.WebHome]] allows for a time zone offset, so that the exact period of time encompassed by the value can be expressed with absolute precision. | ||
| 89 | |||
| 90 | The time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] has a fixed identification which allows for strictly time series formats to be created. These explicit time series formats will be discussed in the [[Data Sets>>doc:sdmx:Glossary.Data set.WebHome]] section. | ||
| 91 | |||
| 92 | The other specialized [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is the [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. A [[measure>>doc:sdmx:Glossary.Measure.WebHome]] can refer to a collection of properties for which phenomena is being measured for an entity classified by the other [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] of the data structure (e.g. the demographic [[measures>>doc:sdmx:Glossary.Measure.WebHome]] in the demography data structure). A [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] might also refer to the different means in which a phenomenum may be measured (e.g. weight, volume, and price for a commodity). | ||
| 93 | |||
| 94 | A [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] must always take its [[representation>>doc:sdmx:Glossary.Representation.WebHome]] from a [[concept scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]]. This [[concept scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] must define a collection of (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) which define the value set of the [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%). An example of this can be seen in the Eurostat Demography data structure. For example, the demography [[measures>>doc:sdmx:Glossary.Measure.WebHome]] [[concept scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] (DEMO_MEASURES) contains only (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) which define demographic measurements. | ||
| 95 | |||
| 96 | An advantage of defining a [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is that it also allows for a more explicit definition of the [[representation>>doc:sdmx:Glossary.Representation.WebHome]] of the observed value for a given [[measure>>doc:sdmx:Glossary.Measure.WebHome]]. This is achieved by defining core [[representations>>doc:sdmx:Glossary.Representation.WebHome]] for the [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%). Note that it is necessary that the primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[representation>>doc:sdmx:Glossary.Representation.WebHome]] in this case be a superset of all possible [[representations>>doc:sdmx:Glossary.Representation.WebHome]] of the [[measure>>doc:sdmx:Glossary.Measure.WebHome]] (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%). This is demonstrated in the Eurostat Demography data structure. An analysis of the demography [[measures>>doc:sdmx:Glossary.Measure.WebHome]] [[concept scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] shows that the number of deaths in a year is measured as an integer, whereas the life expectancy is measured as a [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]] with only one [[decimal>>doc:sdmx:Glossary.Decimals.WebHome]] digit. This [[representation>>doc:sdmx:Glossary.Representation.WebHome]] is carried over to the structure specific schema when the [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is used as the observation [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]], and explicit [[measures>>doc:sdmx:Glossary.Measure.WebHome]] are used. This can be seen in the data structure specific schema for the demography data structure. | ||
| 97 | |||
| 98 | By defining a [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]], a user of the data reported against the data structure will be able to better understand the relationships that exist between the observed values. Another advantage of using a [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is that the data structure can be specific to the [[representation>>doc:sdmx:Glossary.Representation.WebHome]] of observed value because it relates to a specific [[measure>>doc:sdmx:Glossary.Measure.WebHome]] (i.e. it is a (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) in a the [[concept scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] referenced from the [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]). | ||
| 99 | |||
| 100 | ===== //4.3.1.1.3 Attributes// ===== | ||
| 101 | |||
| 102 | A data structure can also define additional [[components>>doc:sdmx:Glossary.Component.WebHome]] which serve to hold additional information (metadata) about the data. This information can be presentational in nature (e.g. a series [[title>>doc:sdmx:Glossary.Title.WebHome]]) or be critical to understanding the data (e.g. the [[unit of measure>>doc:sdmx:Glossary.Unit of measure.WebHome]]). The ECB Exchange Rates data structure contains both of these [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]. | ||
| 103 | |||
| 104 | There is one specialized [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] in a data structure, and this is the reporting year start day. If the time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] of a data structure is allows for reporting periods in its [[representation>>doc:sdmx:Glossary.Representation.WebHome]] then it is strongly recommended that this [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] be used. It has a fixed identifier and [[representation>>doc:sdmx:Glossary.Representation.WebHome]] (a month and day). The purpose of this [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] is to communicate the reference point for a reporting year. This reference point allows the exact calendar period for a reporting period to be determined. If this is not present, then the basis for all reporting periods will be assumed to be January 1. | ||
| 105 | |||
| 106 | In addition to the aforementioned [[component>>doc:sdmx:Glossary.Component.WebHome]] properties, an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] must define its relationship to the other [[components>>doc:sdmx:Glossary.Component.WebHome]] of the data structure (i.e. the [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] or the primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]]). This relationship states how the value of the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] varies with the value of other [[components>>doc:sdmx:Glossary.Component.WebHome]]. In the Eurostat DemographyEurostat Demography data structure, the [[unit of measure>>doc:sdmx:Glossary.Unit of measure.WebHome]] [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] (UNIT_MEASURE) specifies an [[attribute relationship>>doc:sdmx:Glossary.Attribute relationship.WebHome]] with the demographic [[measure>>doc:sdmx:Glossary.Measure.WebHome]] [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] (DEMO). This should be intuitive, since the [[unit of measure>>doc:sdmx:Glossary.Unit of measure.WebHome]] differs if one is measuring a count, such as the number of live births in a year, or a rate, such as the fertility rate. | ||
| 107 | |||
| 108 | A data structure can define groups for the purpose of specifying [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] values. The advantage of defining groups is to avoid repetition of [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] values in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]]. Each group consists of a unique subset of the [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]. [[Attributes>>doc:sdmx:Glossary.Attribute.WebHome]] can either explicitly specify a relationship with the group, or they can specify relationships with specific [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] yet still reference a group for attachment purposes (although the [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] with which the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] have a relationship must all be part of the group [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]). | ||
| 109 | |||
| 110 | === 4.3.2 Data Sets === | ||
| 111 | |||
| 112 | Every [[data set>>doc:sdmx:Glossary.Data set.WebHome]] in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] must conform to a [[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]]. As described above, the [[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]] organizes (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) definitions into various [[components>>doc:sdmx:Glossary.Component.WebHome]] which identify and supplement the data. When processing data, it is critical to be able to retrieved and fully understand the [[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]]. A clear understanding the data structure is dependent on well defined (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%). Therefore, useful data starts with well defined (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%). Ultimately, understanding is dependent upon understanding the (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) which define what is being measured. | ||
| 113 | |||
| 114 | In terms of processing data, the [[data structure definition>>doc:sdmx:Glossary.Data structure definition.WebHome]] provides enough information so that the data can be validated and understood. However, the data structure does not dictate the exact orientation of the data. | ||
| 115 | |||
| 116 | The orientation of the data is defined by the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] itself. However, [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] only allows for two basic orientations: | ||
| 117 | |||
| 118 | 1. A flat list of observations in which the full key is provided for each observed value. | ||
| 119 | 1. A collection of observations in series where all but one [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] has the same value and each observation is distinguished by the other [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] (e.g. a time series in which a series has a key and each observation in the series has a distinct time value). This [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] if known as the observation [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. | ||
| 120 | |||
| 121 | A [[data set>>doc:sdmx:Glossary.Data set.WebHome]] may also contain groups, if the data structure defines them. Each group will have a unique set of key values and provide the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] values associated with the key set. | ||
| 122 | |||
| 123 | [[Data sets>>doc:sdmx:Glossary.Data set.WebHome]] note the orientation by defining the observation [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]]. In the case of the flat orientation, the observation [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is actually all [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]]. In the second case, it is a specific [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] from the data structure (this must be the same [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] for the entire [[data set>>doc:sdmx:Glossary.Data set.WebHome]]). | ||
| 124 | |||
| 125 | This observation [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] dictates where [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] should be present, based on the [[attribute relationships>>doc:sdmx:Glossary.Attribute relationship.WebHome]] defined in the data structure. Any [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] which has a relationship with the observation [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] (i.e. the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] is a part of a group or a set of [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]] with which the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] has an “[[attribute relationship>>doc:sdmx:Glossary.Attribute relationship.WebHome]]”) must exist at the observation level. This also holds true if an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] has a relationship with the primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]]. If an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] has a relationship with no data structure [[components>>doc:sdmx:Glossary.Component.WebHome]], then only one value is provided per [[data set>>doc:sdmx:Glossary.Data set.WebHome]] (i.e. the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] exists at the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] level). | ||
| 126 | |||
| 127 | If an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] has a relationship with a group, or specifies a group for attachment, the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] will be communicated at the group (% style="color:#e74c3c" %)level(%%). In all other cases the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] will exist at the series level. | ||
| 128 | |||
| 129 | Data can be expressed in one of four formats; | ||
| 130 | |||
| 131 | 1. Generic | ||
| 132 | 1. Time series generic | ||
| 133 | 1. Structure specific | ||
| 134 | 1. Time series structure specific | ||
| 135 | |||
| 136 | The time series formats are actually equivalent in content of their more generalized counterparts when they specify the time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] at the observation level. Therefore, with the exception of the root element name, a generic [[data set>>doc:sdmx:Glossary.Data set.WebHome]] with time at the observation level will be the same as a time series generic [[data set>>doc:sdmx:Glossary.Data set.WebHome]]. The difference in these formats is that the time series only allows for time to be the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] at the observation level. | ||
| 137 | |||
| 138 | Note that regardless of the organisation or the format, the data expressed is always the same. This can be seen by examining the various data messages for the ECB Exchange Rates data. Regardless of the organisation or the format, the data expressed is always the same. In fact, this even holds true for the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] values even though they are expressed at different levels depending on their relationships. This serves to show that it is critical that the [[attribute relationship>>doc:sdmx:Glossary.Attribute relationship.WebHome]] be specified correctly, otherwise the correct value cannot be expressed in a data message. | ||
| 139 | |||
| 140 | == 4.4 Metadata == | ||
| 141 | |||
| 142 | [[Reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] enables additional information to be attached to data or [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]]. The design of the [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] model allows informational metadata to be: | ||
| 143 | |||
| 144 | 1. Structured and validated | ||
| 145 | 1. Late bound to the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] or data to which it applies | ||
| 146 | |||
| 147 | Consider [[contact>>doc:Glossary.Contact.WebHome]] information for a [[data set>>doc:sdmx:Glossary.Data set.WebHome]]. This information could be contained is a single data [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] which could be carried in the data message. However, the nature of data [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] is that they have no sub structure. Therefore, there would be no means of separating the name of the contact person from the [[phone number>>doc:sdmx:Glossary.Contact phone number.WebHome]] (outside of creating many [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] to hold this information). In addition, this [[contact>>doc:Glossary.Contact.WebHome]] information is probably subject to change. It would not make much sense to update the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] when the data itself is unchanged simply to specify a new [[contact>>doc:Glossary.Contact.WebHome]]. | ||
| 148 | |||
| 149 | This is where [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] is useful. The structure of [[contact>>doc:Glossary.Contact.WebHome]] information can be clearly specified, and attached dynamically to the data. This same dynamic applies to [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] as well. Although all [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] [[components>>doc:sdmx:Glossary.Component.WebHome]] contain [[annotations>>doc:sdmx:Glossary.Annotation.WebHome]], these often do not allow for the structure that is necessary to communicate the desired information. | ||
| 150 | |||
| 151 | A major use case for [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] is in the support of quality frameworks, where the metadata are not concerned with a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] but with the processes, regulations, and policies of data collection and dissemination. | ||
| 152 | |||
| 153 | === 4.4.1 Defining Metadata Structures === | ||
| 154 | |||
| 155 | [[image:SDMX_2-1_User_Guide_draft_0-1_html_49e79a59c671c22.jpg||data-xwiki-image-style-alignment="center" height="229" width="548"]] | ||
| 156 | |||
| 157 | (% style="text-align: center;" %) | ||
| 158 | **{{id name="image_4"/}}Figure 4: Schematic of the Basic Structure of a Metadata Structure Definition ** | ||
| 159 | |||
| 160 | A metadata structure defines two types of [[component>>doc:sdmx:Glossary.Component.WebHome]] lists. This first type of [[component>>doc:sdmx:Glossary.Component.WebHome]] list, the metadata target, serves to identity the types of objects to which the metadata described by this structure can be attached. The second type of [[component>>doc:sdmx:Glossary.Component.WebHome]] list, the report structure, identifies the content of the metadata reports which can be attached to the target objects. | ||
| 161 | |||
| 162 | This is, in a sense, similar to a data structure. In a data structure the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] list describes how one identifies what is being measured, and the primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]] and [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] describe the details of that measurement. Similarly, the metadata structure uses the metadata target to describe how one identifies what the report pertains to, and the report structure defines the nature of the report in terms of what is to be reported (metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]) . The fundamental difference between the metadata structure and the data structure is that whereas a data structure only has one set of [[dimensions>>doc:sdmx:Glossary.Dimension.WebHome]], [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]], and primary [[measure>>doc:sdmx:Glossary.Measure.WebHome]], a metadata structure can define multiple targets and report structures, and has no [[measures>>doc:sdmx:Glossary.Measure.WebHome]]. | ||
| 163 | |||
| 164 | ==== 4.4.1.1 Metadata Target ==== | ||
| 165 | |||
| 166 | [[image:SDMX_2-1_User_Guide_draft_0-1_html_e8b68e268c2be157.jpg||data-xwiki-image-style-alignment="center" height="370" width="576"]] | ||
| 167 | |||
| 168 | (% style="text-align: center;" %) | ||
| 169 | **{{id name="image_"/}}Figure 5: Identification of Targets in a Metadata Structure Definition** | ||
| 170 | |||
| 171 | A metadata target defines what is expected in the [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]] in order to identify the object to which the metadata pertains. It is given a unique identifier within the metadata structure in which it is defined. It consists of one or more target object descriptors, each of which themselves have a unique identifier within the metadata target. | ||
| 172 | |||
| 173 | There are 5 types of target objects that one can use, each of which serves to uniquely identify an object within the [[SDMX information model>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]. | ||
| 174 | |||
| 175 | ===== //4.4.1.1.1 Data Set Target// ===== | ||
| 176 | |||
| 177 | The [[data set>>doc:sdmx:Glossary.Data set.WebHome]] target is used to attach metadata to a specific [[data set>>doc:sdmx:Glossary.Data set.WebHome]], which is identified by the identification of the [[data provider>>doc:sdmx:Glossary.Data provider.WebHome]] and the provider assigned identification of the [[data set>>doc:sdmx:Glossary.Data set.WebHome]]. This target object has a fixed identifier and [[representation>>doc:sdmx:Glossary.Representation.WebHome]], so the purpose of defining this in a metadata target is to simply state that the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] reference is part of the metadata target value set. Only one [[data set>>doc:sdmx:Glossary.Data set.WebHome]] target can occur within a metadata target. | ||
| 178 | |||
| 179 | ===== //4.4.1.1.2 Identifiable Object Target// ===== | ||
| 180 | |||
| 181 | The identifiable target object is used to attach metadata to any identifiable object in the [[SDMX information model>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]. This type of target object can be repeated and each instance is assigned a unique identifier within the metadata target. Each instance identifies the type of object which is reference by this target. The identification of the target object is always a complete reference. If the target identifiable object type is an item from within an [[item scheme>>doc:sdmx:Glossary.Item scheme.WebHome]], the [[representation>>doc:sdmx:Glossary.Representation.WebHome]] of the target object can reference a scheme for the purposes of limiting the items to which metadata can be attached. | ||
| 182 | |||
| 183 | ===== //4.4.1.1.3 Dimension Descriptor Values Target// ===== | ||
| 184 | |||
| 185 | The key descriptor values target is used to attach metadata to data by identifying full or partial data "keys" (a collection of [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] identifier/value pairs). By itself, this target object is typically not descriptive enough as it does not identify the type of data to which the keys apply. Therefore, this is typically used with other target objects, such as the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] or data structure, which can identify these data. This target object has a fixed identifier and [[representation>>doc:sdmx:Glossary.Representation.WebHome]], therefore the metadata target is simply stating that it contains a key descriptor value set. Only one key descriptor values target can occur within a metadata target. | ||
| 186 | |||
| 187 | ===== //4.4.1.1.4 Constraint Content Target// ===== | ||
| 188 | |||
| 189 | The [[constraint>>doc:Glossary.Constraint.WebHome]] content target is used to attach metadata to data by referencing an attachment [[constraint>>doc:Glossary.Constraint.WebHome]]. This attachment [[constraint>>doc:Glossary.Constraint.WebHome]] turn defines the [[data set>>doc:sdmx:Glossary.Data set.WebHome]](s) and keys to which the metadata applies. This is equivalent to using the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] or identifiable object target along with the key descriptor values target. The difference is that the attachment [[constraint>>doc:Glossary.Constraint.WebHome]] allows the target set to be defined once, and be reused by multiple reports, whereas the former method would require that the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] (or equivalent data structure, [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], or [[provision agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]]) reference and key descriptor value set be repeated for each report. Only one [[constraint>>doc:Glossary.Constraint.WebHome]] content target can occur within a metadata target. | ||
| 190 | |||
| 191 | ===== //4.4.1.1.5 Report Period Target// ===== | ||
| 192 | |||
| 193 | The report period target is used to state the reporting period for which metadata is applicable. This effectively allows the metadata to change over time while persisting the historicity of the changes. This target object has a fixed identifier, but its [[representation>>doc:sdmx:Glossary.Representation.WebHome]] can specify the specific type of date that can be used. By default, this is the least restrictive date format. Only one report period target can occur within a metadata target. | ||
| 194 | |||
| 195 | ===== //4.4.1.1.6 Composing Targets// ===== | ||
| 196 | |||
| 197 | A metadata target consists of one or more of the target objects described above. It is the sum of these targets which identify the actual target for the metadata. For example, one might attach metadata to a specific portion of data in a given [[data set>>doc:sdmx:Glossary.Data set.WebHome]]. In this case, the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] target object and the key descriptor values target object would be used. In the [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]], the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] target would identify the [[data set>>doc:sdmx:Glossary.Data set.WebHome]] and the key descriptor values target would identify the portions of the data to which the metadata applies. In another example, one might wish to be able to attach metadata to portions of data for all [[data sets>>doc:sdmx:Glossary.Data set.WebHome]] reported against a given data structure. Rather than repeating the metadata for each [[data set>>doc:sdmx:Glossary.Data set.WebHome]], a metadata target would be defined which contains an identifiable target object which references a data structure and a key descriptor values target object. When designing a metadata target one must consider how generally the metadata which is reported against the target might be applied. | ||
| 198 | |||
| 199 | ==== 4.4.1.2 Report Structure ==== | ||
| 200 | |||
| 201 | [[image:SDMX_2-1_User_Guide_draft_0-1_html_bb561ad102f99f98.jpg||data-xwiki-image-style-alignment="center" height="502" width="575"]] | ||
| 202 | |||
| 203 | (% style="text-align: center;" %) | ||
| 204 | **{{id name="image_6"/}}Figure 6: Schematic of the Report Structure in a Metadata Structure Definition** | ||
| 205 | |||
| 206 | The metadata structure defines one or more report structures which define the content of its metadata reports. Each report structure is assigned a unique identifier within the metadata structure. The report structure defines the metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] which make up its content and references the metadata targets from within the metadata structure that define where the report can be attached. Since a [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]] can only contain metadata reports for a single metadata structure, one must consider whether having reports in the same [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]] would be useful. Note that all reports in a [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]] must be defined in the same metadata structure. | ||
| 207 | |||
| 208 | ===== //4.4.1.2.1 Metadata Attribute// ===== | ||
| 209 | |||
| 210 | A metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] is [[component>>doc:sdmx:Glossary.Component.WebHome]] of the content of a metadata report. Similar to a data [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] in a data structure, the metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] takes its semantic from a (% style="color:#e74c3c" %)concept(%%). This (% style="color:#e74c3c" %)concept(%%) serves to define the meaning of the information contained in the report. As with data structure [[components>>doc:sdmx:Glossary.Component.WebHome]], it is important to make use of common (% style="color:#e74c3c" %)concepts(%%) whenever possible, as this makes the metadata relatable to other metadata reports. | ||
| 211 | |||
| 212 | The content of a metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] can be a value and/or other metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]. The value of a metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] can serve a number of purposes. First, as with a data [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] it can be an enumerated value from a [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[code list>>doc:sdmx:Glossary.Code list.WebHome]]. It can also be a noncoded value of any given data type. Where the metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] value differs from that of a data [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] is that if the value is textual, it can be represented in parallel language values, and if necessary be structured using XHTML. It is not necessary that a metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] has a value, as it might only serve to contain other metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] for the purpose of organising metadata reports (in which case the metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] is designated as “isPresentational” indicating that no value is expected to be reported for the metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] in a report).l. | ||
| 213 | |||
| 214 | Metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] are not reusable across various (% style="color:#e74c3c" %)levels(%%) of a report structure. If an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] is intended to occur at multiple (% style="color:#e74c3c" %)levels(%%), it must be redefined at each (% style="color:#e74c3c" %)level(%%). However, the identification of a metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] is only required to be unique within the scope within which it is defined. Therefore, if the intention is to reuse a metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] at different (% style="color:#e74c3c" %)levels(%%), it is recommended that the same identification be used to convey this intention. It should also be noted that within a scope, a metadata [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] can have cardinality (minimum and maximum number of occurrences). This allows metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] to be repeated at various (% style="color:#e74c3c" %)levels(%%), as well as giving the report structure the ability to enforce content requirements. | ||
| 215 | |||
| 216 | === 4.4.2 Metadata Sets === | ||
| 217 | |||
| 218 | A [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]] is a collection of metadata reports from the same [[metadata structure definition>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]. Each report is based on a report structure defined within the metadata structure on which the set is based. More than one report for a given report structure can exist within a set, so long as their targets are unique, which is to say that for any given target there should only be one instance of a report for a given report structure. | ||
| 219 | |||
| 220 | The manner in which a metadata report is related to an object is through the specification of its target. The metadata structure defines the possible types of targets for a given report structure, and an instance of a report uses one of these target types to identify the actual target of the metadata. The target is essentially a collection of references to data or [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] and possibly a period to which the report applies. It is assumed that any system processing metadata reports will be able to resolve these references, or perhaps more appropriately, any system working with data or [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] will be able to process the related metadata reports. | ||
| 221 | |||
| 222 | The actual content of the report is always contained in an [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] set. This [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] set is the collection of metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] for which values are provided. As with data, the key to a useful metadata report is understanding what is being reported. This comes down to effective (% style="color:#e74c3c" %)concept(%%) usage. The content of any report is essentially made up of values reported against (% style="color:#e74c3c" %)concepts(%%). In order for systems to understand the meaning of the metadata, they must understand the (% style="color:#e74c3c" %)concepts(%%). | ||
| 223 | |||
| 224 | With the [[metadata structure definition>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], the content of any report can be evaluated for completeness and validity. From the metadata structure, one can determine if all necessary metadata [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] are present for a given report structure and if the values assigned to them are allowable based on the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] definition. | ||
| 225 | |||
| 226 | In the Eurostat DemographyEurostat Demography metadata samples, one can see the similarities between the generic and the structure specific messages. The content of the reported metadata does not change with the format used. The structure specific metadata simply provides more validation of the content. |