Skip to Content
Last modified by Artur on 2025/09/10 11:19
From version 1.1
edited by Artur
on 2025/04/21 12:12
Change comment: Imported from XAR
To version 5.1
edited by Artur
on 2025/07/14 10:19
Change comment: Imported from XAR

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -Methodology.SDMX 2\.1 User Guide.WebHome
1 +WebHome
Tags
... ... @@ -1,0 +1,1 @@
1 +Artefact|Category scheme|Code list|Concept scheme|Data consumer|Data provider|Data set|Data structure definition|Dataflow|Dimension|Hub (dissemination architecture)|Maintenance agency|Measure|Metadata repository|Metadata set|Metadata structure definition|Metadataflow|Notification|Provision agreement|Reference metadata|SDMX Information Model|SDMX Registry|SDMX-ML|Statistical data and metadata exchange|Statistical subject-matter domain|Structural metadata|Subscription
Content
... ... @@ -4,42 +4,40 @@
4 4  
5 5  == 8.1 Scope of this Chapter ==
6 6  
7 -(% style="text-align: center;" %)
8 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_4c1c6e5debb65e85.png||height="411" width="679"]]
7 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_4c1c6e5debb65e85.png||data-xwiki-image-style-alignment="center" height="411" width="679"]]
9 9  
10 -In this guide it has been assumed that the structural metadata (e.g. DSD, MSD, Code List, Concept Scheme) is available to processing applications such as a data extract function of a database or a data visualization function of a website. For many SDMX processing applications these structural metadata must be made available “on demand”.
9 +In this guide it has been assumed that the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] (e.g. [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], [[Code List>>doc:sdmx:Glossary.Code list.WebHome]], [[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]]) is available to processing applications such as a data extract function of a database or a data visualization function of a website. For many [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] processing applications these [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] must be made available “on demand”.
11 11  
12 -This Chapter explains the role and functions of an SDMX Registry and how the Registry is used both as a storage, retrieval, and maintenance repository for SDMX structural metadata, and as a mechanism for discovering data and reference metadata including the automated “pull” scenario of an automated data reporting system .
11 +This Chapter explains the role and functions of an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] and how the Registry is used both as a storage, retrieval, and maintenance repository for [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]], and as a mechanism for discovering data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] including the automated “pull” scenario of an automated data reporting system .
13 13  
14 14  == 8.2 The Need for an SDMX Registry ==
15 15  
16 -First, it should be stated that it is not necessary to operate or have access to an SDMX Registry in order to use SDMX. Many use cases do, however, require access to structural metadata but these can be retrieved from any web service that can respond to a query for structural metadata (either the REST query or the “Structure Where” query) - this service does not need to be an SDMX Registry.
15 +First, it should be stated that it is not necessary to operate or have access to an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] in order to use [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]. Many use cases do, however, require access to [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] but these can be retrieved from any web service that can respond to a query for [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] (either the REST query or the “Structure Where” query) - this service does not need to be an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]].
17 17  
18 -Whilst both an SDMX Registry and a non-Registry web service use exactly the same query mechanism, a Registry differs from non-Registry web service in three important areas:
17 +Whilst both an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] and a non-Registry web service use exactly the same query mechanism, a Registry differs from non-Registry web service in three important areas:
19 19  
20 -1. The Registry offers a maintenance service for structural metadata – this is the “Repository” function of the Registry.
19 +1. The Registry offers a maintenance service for [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] – this is the “Repository” function of the Registry.
21 21  1. The Registry offers a data and metadata Registration service
22 -1. The Registry offers a subscription and notification service
21 +1. The Registry offers a [[subscription>>doc:sdmx:Glossary.Subscription.WebHome]] and [[notification>>doc:sdmx:Glossary.Notification.WebHome]] service
23 23  
24 24  The functions of these services are covered in this Chapter.
25 25  
26 -Software tools that support SDMX Registry services are becoming more functional and have reached a point where it is extremely easy to install and operate an SDMX Registry. This can be used to support a community of users or be used solely within a single organization in order to have a central repository for SDMX structural metadata which can be maintained and retrieved.
25 +Software tools that support [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] services are becoming more functional and have reached a point where it is extremely easy to install and operate an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]]. This can be used to support a community of users or be used solely within a single organization in order to have a central repository for [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] which can be maintained and retrieved.
27 27  
28 28  == 8.3 Objective of a Registry ==
29 29  
30 -The objective of the SDMX registry/repository is, in broad terms, to allow organisations to publish statistical data and reference metadata in known formats such that interested third parties can discover these data and interpret them accurately and correctly. The mechanism for doing this is twofold:
29 +The objective of the [[SDMX registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]]/repository is, in broad terms, to allow organisations to publish statistical data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] in known formats such that interested third parties can discover these data and interpret them accurately and correctly. The mechanism for doing this is twofold:
31 31  
32 -1. To maintain and publish structural metadata that describes the structure and valid content of data and reference metadata sources such as databases, metadata repositories, data sets, metadata sets. This structural metadata enables software applications to understand and to interpret the data and reference metadata in these sources.
33 -1. To enable applications, organisations, and individuals to share and to discover data and reference metadata. This facilitates data and reference metadata dissemination by implementing the data sharing vision of SDMX.
31 +1. To maintain and publish [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] that describes the structure and valid content of data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] sources such as databases, [[metadata repositories>>doc:sdmx:Glossary.Metadata repository.WebHome]], [[data sets>>doc:sdmx:Glossary.Data set.WebHome]], [[metadata sets>>doc:sdmx:Glossary.Metadata set.WebHome]]. This [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] enables software applications to understand and to interpret the data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] in these sources.
32 +1. To enable applications, organisations, and individuals to share and to discover data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]]. This facilitates data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] dissemination by implementing the data sharing vision of [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].
34 34  
35 35  == 8.4 SDMX Registry/Repository Architecture ==
36 36  
37 37  === 8.4.1 Architectural Schematic ===
38 38  
39 -The architecture of the SDMX registry/repository is a layered architecture that is founded by a structural metadata repository which supports a provisioning metadata repository which supports the registry services. These are all supported by the SDMX-ML schemas. Applications can be built on top of these services which support the reporting, storage, retrieval, and dissemination aspects of the statistical lifecycle as well as the maintenance of the structural metadata required to drive these applications.
38 +The architecture of the [[SDMX registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]]/repository is a layered architecture that is founded by a [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] repository which supports a provisioning [[metadata repository>>doc:sdmx:Glossary.Metadata repository.WebHome]] which supports the registry services. These are all supported by the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] schemas. Applications can be built on top of these services which support the reporting, storage, retrieval, and dissemination aspects of the statistical lifecycle as well as the maintenance of the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] required to drive these applications.
40 40  
41 -(% style="text-align: center;" %)
42 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_811113a255dfd7e.jpg||height="325" width="554"]]
40 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_811113a255dfd7e.jpg||data-xwiki-image-style-alignment="center" height="325" width="554"]]
43 43  
44 44  (% style="text-align: center;" %)
45 45  **{{id name="image_19"/}}Figure 19: Schematic of the Registry Content and Services**
... ... @@ -46,18 +46,18 @@
46 46  
47 47  === 8.4.2 Structural Metadata Repository ===
48 48  
49 -The basic layer is that of a structural metadata service which supports the lifecycle of SDMX structural metadata artefacts such as Maintenance Agencies, Data Structure Definitions, Metadata Structure Definitions, Provision Agreements, Processes etc. This layer is supported by the Structure Maintenance and Query Service.
47 +The basic layer is that of a [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] service which supports the lifecycle of [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] such as [[Maintenance Agencies>>doc:sdmx:Glossary.Maintenance agency.WebHome]], [[Data Structure Definitions>>doc:sdmx:Glossary.Data structure definition.WebHome]], [[Metadata Structure Definitions>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], [[Provision Agreements>>doc:sdmx:Glossary.Provision agreement.WebHome]], Processes etc. This layer is supported by the Structure Maintenance and Query Service.
50 50  
51 -Note that the SDMX-ML Submit Structure Request message supports all of the SDMX structural artefacts. The only Registry artefacts that are not supported by the SDMX-ML Submit Structure Request are:
49 +Note that the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] Submit Structure Request message supports all of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] structural [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]]. The only Registry [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]] that are not supported by the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] Submit Structure Request are:
52 52  
53 53  * Registration of data and metadata sources
54 -* Subscription and Notification
52 +* [[Subscription>>doc:sdmx:Glossary.Subscription.WebHome]] and [[Notification>>doc:sdmx:Glossary.Notification.WebHome]]
55 55  
56 -Separate registry-based messages are defined to support these artefacts.
54 +Separate registry-based messages are defined to support these [[artefacts>>doc:sdmx:Glossary.Artefact.WebHome]].
57 57  
58 58  === 8.4.3 Provisioning Metadata Repository ===
59 59  
60 -The function of this repository is to support the registration of various types of data-store which model SDMX-conformant databases or files, and to link to these data and reference metadata sources. These links can be specified for a data provider, for a specific data or metadata flow. In the SDMX model this is called the Provision Agreement.
58 +The function of this repository is to support the registration of various types of data-store which model [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-conformant databases or files, and to link to these data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] sources. These links can be specified for a [[data provider>>doc:sdmx:Glossary.Data provider.WebHome]], for a specific data or [[metadata flow>>doc:sdmx:Glossary.Metadataflow.WebHome]]. In the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] model this is called the [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]].
61 61  
62 62  This layer is supported by the Data and Metadata Registration Service.
63 63  
... ... @@ -65,48 +65,45 @@
65 65  
66 66  === 8.5.1 Structure Maintenance and Query Service ===
67 67  
68 -A Registry offers a web service for the maintenance of structural metadata. Whilst this service must comply with the SDMX-ML Registry Interface, it is common for such registries to also provide a graphical user interface for the submission and maintenance of SDMX structures.
66 +A Registry offers a web service for the maintenance of [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]]. Whilst this service must comply with the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] Registry Interface, it is common for such registries to also provide a graphical user interface for the submission and maintenance of [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] structures.
69 69  
70 -(% style="text-align: center;" %)
71 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_1441cca2307b86ac.jpg||height="279" width="576"]]
68 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_1441cca2307b86ac.jpg||data-xwiki-image-style-alignment="center" height="279" width="576"]]
72 72  
73 73  (% style="text-align: center;" %)
74 74  **{{id name="image_20"/}}Figure 20: Example Graphical User Interface to an SDMX Registry **
75 75  
76 -The Registry must also offer a query service for structural metadata.
73 +The Registry must also offer a query service for [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]].
77 77  
78 78  === 8.5.2 Registration Service ===
79 79  
80 -The Registration service allows a data or metadata publisher (in SDMX terminology this is a Data Provider) to publish that data or metadata exist. As the Registration is connected to a Provision Agreement which itself is connected to a Data Provider and Data or Metadata Flow, the exact type of data or metadata registered is known, and also its structure (as the Data or Metadata Flow references the DSD or MSD).
77 +The Registration service allows a data or metadata publisher (in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] terminology this is a [[Data Provider>>doc:sdmx:Glossary.Data provider.WebHome]]) to publish that data or metadata exist. As the Registration is connected to a [[Provision Agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]] which itself is connected to a [[Data Provider>>doc:sdmx:Glossary.Data provider.WebHome]] and Data or [[Metadata Flow>>doc:sdmx:Glossary.Metadataflow.WebHome]], the exact type of data or metadata registered is known, and also its structure (as the Data or [[Metadata Flow>>doc:sdmx:Glossary.Metadataflow.WebHome]] references the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] or [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]).
81 81  
82 -The Registration must include a URL of either a file of SDMX-ML containing a data or metadata set, or a URL of a web service which can be queried to obtain the data.
79 +The Registration must include a URL of either a file of [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] containing a data or [[metadata set>>doc:sdmx:Glossary.Metadata set.WebHome]], or a URL of a web service which can be queried to obtain the data.
83 83  
84 -(% style="text-align: center;" %)
85 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_541961e6279194a1.jpg||height="446" width="451"]]
81 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_541961e6279194a1.jpg||data-xwiki-image-style-alignment="center" height="446" width="451"]]
86 86  
87 87  (% style="text-align: center;" %)
88 88  **{{id name="image_21"/}}Figure 21: Schematic of Registered Data and Metadata Sources in the SDMX-IM**
89 89  
90 -The Registration can be used both for data consumers to find data (this is described below in “Data and Metadata Discovery”) and for automated data reporting systems to harvest the data (the “pull” scenario).
86 +The Registration can be used both for [[data consumers>>doc:sdmx:Glossary.Data consumer.WebHome]] to find data (this is described below in “Data and Metadata Discovery”) and for automated data reporting systems to harvest the data (the “pull” scenario).
91 91  
92 92  === 8.5.3 Subscription and Notification Service ===
93 93  
94 -This subscription service allows a user to monitor Registry events. An event is something that has changed in the Registry. The subscription can be specified for specific objects such as a new registration of data or a change to a code list, or it can be specified for many objects such as any change to a specific type of structure or all registrations. A subscription can ask for notification by e-mail or notification to a URL (this will be a web service that can act upon the notification such as the automated update of structural metadata supporting a web dissemination service or the “pull” mechanism for data reporting).
90 +This [[subscription>>doc:sdmx:Glossary.Subscription.WebHome]] service allows a user to monitor Registry events. An event is something that has changed in the Registry. The [[subscription>>doc:sdmx:Glossary.Subscription.WebHome]] can be specified for specific objects such as a new registration of data or a change to a [[code list>>doc:sdmx:Glossary.Code list.WebHome]], or it can be specified for many objects such as any change to a specific type of structure or all registrations. A [[subscription>>doc:sdmx:Glossary.Subscription.WebHome]] can ask for [[notification>>doc:sdmx:Glossary.Notification.WebHome]] by e-mail or [[notification>>doc:sdmx:Glossary.Notification.WebHome]] to a URL (this will be a web service that can act upon the [[notification>>doc:sdmx:Glossary.Notification.WebHome]] such as the automated update of [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] supporting a web dissemination service or the “pull” mechanism for data reporting).
95 95  
96 -When a monitored event is triggered the notification service notifies interested parties of the Registry content that has changed. The notification can be user (email) or application (URL).
92 +When a monitored event is triggered the [[notification>>doc:sdmx:Glossary.Notification.WebHome]] service notifies interested parties of the Registry content that has changed. The [[notification>>doc:sdmx:Glossary.Notification.WebHome]] can be user (email) or application (URL).
97 97  
98 98  == 8.6 Data and Metadata Discovery ==
99 99  
100 100  === 8.6.1 Choreography ===
101 101  
102 -Discovering published data and reference metadata involves interaction with the Registry to:
98 +Discovering published data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] involves interaction with the Registry to:
103 103  
104 -* optionally (but usually) browsing a subject matter domain category scheme to find Dataflow Definitions (and hence Data Structure Definitions) and Metadataflows (and hence Metadata Structure Definitions) which structure the type of data and/or reference metadata being sought
105 -* build a query, in terms of the selected Data Structure Definition or Metadata Structure Definition, which specifies what data are required and submitting this to a service that can query an SDMX Registry which will return a list of (URLs of) data and reference metadata files and databases which satisfy the query
106 -* processing the query result set and retrieving data and/or reference metadata from the supplied URLs
100 +* optionally (but usually) browsing a subject matter domain [[category scheme>>doc:sdmx:Glossary.Category scheme.WebHome]] to find [[Dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] Definitions (and hence [[Data Structure Definitions>>doc:sdmx:Glossary.Data structure definition.WebHome]]) and [[Metadataflows>>doc:sdmx:Glossary.Metadataflow.WebHome]] (and hence [[Metadata Structure Definitions>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]) which structure the type of data and/or [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] being sought
101 +* build a query, in terms of the selected [[Data Structure Definition>>doc:sdmx:Glossary.Data structure definition.WebHome]] or [[Metadata Structure Definition>>doc:sdmx:Glossary.Metadata structure definition.WebHome]], which specifies what data are required and submitting this to a service that can query an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] which will return a list of (URLs of) data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] files and databases which satisfy the query
102 +* processing the query result set and retrieving data and/or [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] from the supplied URLs
107 107  
108 -(% style="text-align: center;" %)
109 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_51f6d84398cd8ba3.jpg||height="453" width="510"]]
104 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_51f6d84398cd8ba3.jpg||data-xwiki-image-style-alignment="center" height="453" width="510"]]
110 110  
111 111  (% style="text-align: center;" %)
112 112  **{{id name="image_22"/}}Figure 22: Schematic of Data and Metadata Discovery and Query in the SDMX-IM**
... ... @@ -117,13 +117,11 @@
117 117  
118 118  == 8.7 Patterns for Data and Metadata Exchange ==
119 119  
120 -SDMX identifies three basic process patterns and two modes (push and pull) regarding the exchange of statistical data and metadata.
115 +[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] identifies three basic process patterns and two modes (push and pull) regarding the exchange of statistical data and metadata.
121 121  
122 -(% style="text-align: center;" %)
123 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_b247734721100020.png||height="547" width="680"]]
117 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_b247734721100020.png||data-xwiki-image-style-alignment="center" height="547" width="680"]]
124 124  
125 -(% style="text-align: center;" %)
126 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_2a1b6d63f93d6579.png||height="300" width="680"]]
119 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_2a1b6d63f93d6579.png||data-xwiki-image-style-alignment="center" height="300" width="680"]]
127 127  
128 128  (% style="text-align: center;" %)
129 129  **{{id name="image_23"/}}Figure 23: The Three Basic Patterns of Data and Reference Metadata Exchange**
... ... @@ -130,39 +130,38 @@
130 130  
131 131  In the //push// mode the data/metadata are sent by the reporter to the collector.
132 132  
133 -In the //pull// mode the data consumer retrieves the data from the reporter's web server. The data may be made available for download in an SDMX-conformant file, or they may be retrieved from a database in response to an SDMX-conformant query, via a web service running on the reporter's server. In both cases, the data are made available to any organisation requiring them, in formats which ensure that data are consistently described by appropriate metadata, whose meaning is common to all parties in the exchange.
126 +In the //pull// mode the [[data consumer>>doc:sdmx:Glossary.Data consumer.WebHome]] retrieves the data from the reporter's web server. The data may be made available for download in an [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-conformant file, or they may be retrieved from a database in response to an [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-conformant query, via a web service running on the reporter's server. In both cases, the data are made available to any organisation requiring them, in formats which ensure that data are consistently described by appropriate metadata, whose meaning is common to all parties in the exchange.
134 134  
135 -Data sharing using the pull mode is well adapted to the database-driven and data hub architectures. Both architectures provide the best benefits for the data producers because they can lessen the burden of publishing the data to multiple counterparties.
128 +Data sharing using the pull mode is well adapted to the database-driven and data [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] architectures. Both architectures provide the best benefits for the data producers because they can lessen the burden of publishing the data to multiple counterparties.
136 136  
137 -In both architectures, it is necessary to implement a notification mechanism, providing provisioning metadata in order to alert collecting organisations that data and metadata sets are made available by data providers, details about the online mechanism for getting data (for example, a queryable online database or a simple URL) and constraints regarding the allowable content of the data sets that will be provided.
130 +In both architectures, it is necessary to implement a [[notification>>doc:sdmx:Glossary.Notification.WebHome]] mechanism, providing provisioning metadata in order to alert collecting organisations that data and [[metadata sets>>doc:sdmx:Glossary.Metadata set.WebHome]] are made available by [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]], details about the online mechanism for getting data (for example, a queryable online database or a simple URL) and (% style="color:#e74c3c" %)constraints(%%) regarding the allowable content of the [[data sets>>doc:sdmx:Glossary.Data set.WebHome]] that will be provided.
138 138  
139 -At the heart of a data-sharing architecture there is often an SDMX Registry. This is a central location where structural and provisioning metadata can be found. In fact all the users/applications that need to access data can query the registry in order to know what data sets and metadata sets are available from data providers, and how to access them.
132 +At the heart of a data-sharing architecture there is often an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]]. This is a central location where structural and provisioning metadata can be found. In fact all the users/applications that need to access data can query the registry in order to know what [[data sets>>doc:sdmx:Glossary.Data set.WebHome]] and [[metadata sets>>doc:sdmx:Glossary.Metadata set.WebHome]] are available from [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]], and how to access them.
140 140  
141 141  === 8.7.1 The Database-driven Architecture ===
142 142  
143 -The database-driven architecture is implemented by those collecting organisations that periodically need to fetch the data and to load them in their database. In general a batch process is used in order to automate the flow in which a whole or a partial dataset, including incremental updating, is used.
136 +The database-driven architecture is implemented by those collecting organisations that periodically need to fetch the data and to load them in their database. In general a batch process is used in order to automate the flow in which a whole or a partial [[dataset>>doc:sdmx:Glossary.Data set.WebHome]], including incremental updating, is used.
144 144  
145 145  From the data management point of view, the pull approach within a database-driven architecture includes the following steps:
146 146  
147 -1) when new data are available, the data provider should:
140 +1) when new data are available, the [[data provider>>doc:sdmx:Glossary.Data provider.WebHome]] should:
148 148  
149 -a) create an SDMX-ML file containing the new data set
142 +a) create an [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] file containing the new data set
150 150  
151 151  __**or**__
152 152  
153 -b) provide a web service (WS) that builds SDMX-ML messages upon request.
146 +b) provide a web service (WS) that builds [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] messages upon request.
154 154  
155 -In both cases a provision agreement must be in place.
148 +In both cases a [[provision agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]] must be in place.
156 156  
157 -2) the data collector Pull Requestor is notified of the new registration. Note that it is common for a Registry to provide an RSS mechanism, though this is nit a formal part of the SDMX standard. The data collector then:
150 +2) the data collector Pull Requestor is notified of the new registration. Note that it is common for a Registry to provide an RSS mechanism, though this is nit a formal part of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] standard. The data collector then:
158 158  
159 -a) retrieves the SDMX-ML file from the specified URL, if it resides in a URL,
152 +a) retrieves the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] file from the specified URL, if it resides in a URL,
160 160  __**or**__
161 161  
162 -b) uses the Query Message or REST Query included in the feed to query the data provider web service, if the data are prepared by the data provider web service.
155 +b) uses the Query Message or REST Query included in the feed to query the [[data provider>>doc:sdmx:Glossary.Data provider.WebHome]] web service, if the data are prepared by the [[data provider>>doc:sdmx:Glossary.Data provider.WebHome]] web service.
163 163  
164 -(% style="text-align: center;" %)
165 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_ba1159b7bd2e98c4.png||height="410" width="469"]]
157 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_ba1159b7bd2e98c4.png||data-xwiki-image-style-alignment="center" height="410" width="469"]]
166 166  
167 167  (% style="text-align: center;" %)
168 168  **{{id name="image_24"/}}Figure 24: Database-driven Architecture**
... ... @@ -169,29 +169,28 @@
169 169  
170 170  === 8.7.2 The Data Hub Architecture ===
171 171  
172 -The data hub architecture consists of an accessible system providing involved actors with the following services:
164 +The data [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] architecture consists of an accessible system providing involved actors with the following services:
173 173  
174 -* data providers can:
166 +* [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]] can:
175 175  
176 -*
177 -** notify the hub of new sets of data and corresponding structural metadata (measures, dimension, code lists, etc.);
168 +*
169 +** notify the [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] of new sets of data and corresponding [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] ([[measures>>doc:sdmx:Glossary.Measure.WebHome]], [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]], [[code lists>>doc:sdmx:Glossary.Code list.WebHome]], etc.);
178 178  ** make data available directly from their systems through a querying system.
179 179  
180 180  * data users can:
181 181  
182 -*
183 -** browse the hub to define a dataset of interest via the above structural metadata;
184 -** retrieve the dataset from the data providers.
174 +*
175 +** browse the [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] to define a [[dataset>>doc:sdmx:Glossary.Data set.WebHome]] of interest via the above [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]];
176 +** retrieve the [[dataset>>doc:sdmx:Glossary.Data set.WebHome]] from the [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]].
185 185  
186 -From the data management point of view, the hub is also based on agreed hypercubes or datasets, but here the hypercubes or datasets are not sent to the central system. Instead the following process operates:
178 +From the data management point of view, the [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] is also based on agreed hypercubes or [[datasets>>doc:sdmx:Glossary.Data set.WebHome]], but here the hypercubes or [[datasets>>doc:sdmx:Glossary.Data set.WebHome]] are not sent to the central system. Instead the following process operates:
187 187  
188 -1. a user identifies a dataset through the web interface of the central hub using the structural metadata, and requests it;
189 -1. the central hub translates the user request in one or more queries and sends them to the related data providers’ systems;
190 -1. data providers’ systems process the query and send the result to the central hub in a standard format;
191 -1. the central hub puts together all the results originated by all interested data providers’ systems and presents them in a human readable format.
180 +1. a user identifies a [[dataset>>doc:sdmx:Glossary.Data set.WebHome]] through the web interface of the central [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] using the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]], and requests it;
181 +1. the central [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] translates the user request in one or more queries and sends them to the related [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]]’ systems;
182 +1. [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]]’ systems process the query and send the result to the central [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] in a standard format;
183 +1. the central [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] puts together all the results originated by all interested [[data providers>>doc:sdmx:Glossary.Data provider.WebHome]]’ systems and presents them in a human readable format.
192 192  
193 -(% style="text-align: center;" %)
194 -[[image:SDMX_2-1_User_Guide_draft_0-1_html_1c9984b4c53f1c50.png||height="335" width="383"]]
185 +[[image:SDMX_2-1_User_Guide_draft_0-1_html_1c9984b4c53f1c50.png||data-xwiki-image-style-alignment="center" height="335" width="383"]]
195 195  
196 196  (% style="text-align: center;" %)
197 197  **{{id name="image_25"/}}Figure 25: Data Hub Architecture**
... ... @@ -198,24 +198,24 @@
198 198  
199 199  === 8.7.3 Data Producer Architectures ===
200 200  
201 -In order to implement an SDMX IT architecture for data-sharing using the pull mode, several steps must be accomplished by a data producer and several questions must be considered:
192 +In order to implement an [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] IT architecture for data-sharing using the pull mode, several steps must be accomplished by a data producer and several questions must be considered:
202 202  
203 -1. which statistical domains are involved and where are the data currently stored?
204 -1. which structural metadata are involved, and where are they currently stored?
205 -1. what is the business process behind the data flow involved in the exercise?
206 -1. will the SDMX data producer architecture be part of a data warehouse architecture, of a data hub architecture or of both?
194 +1. which [[statistical domains>>doc:sdmx:Glossary.Statistical subject-matter domain.WebHome]] are involved and where are the data currently stored?
195 +1. which [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] are involved, and where are they currently stored?
196 +1. what is the business process behind the [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]] involved in the exercise?
197 +1. will the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] data producer architecture be part of a data warehouse architecture, of a data [[hub>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] architecture or of both?
207 207  
208 -Generally data and structural metadata that will be involved in the new SDMX information system are stored either in databases or in files. The two cases lead to different architectural approaches:
199 +Generally data and [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] that will be involved in the new [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] information system are stored either in databases or in files. The two cases lead to different architectural approaches:
209 209  
210 -* a. data and structural metadata are already stored in a database and it is necessary to build suitable software interfaces in order to make the system “SDMX-compliant”.
211 -* b. a separate special-purpose database is set up to store data and structural metadata. This database will be designed with the main aim of being part of an SDMXcompliant system. In this case the database can be modelled using the SDMX Information Model. An SDMX Registry is, of course, such a database.
201 +* a. data and [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] are already stored in a database and it is necessary to build suitable software interfaces in order to make the system “[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-compliant”.
202 +* b. a separate special-purpose database is set up to store data and [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]]. This database will be designed with the main aim of being part of an SDMXcompliant system. In this case the database can be modelled using the [[SDMX Information Model>>doc:sdmx:Glossary.SDMX Information Model.WebHome]]. An [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] is, of course, such a database.
212 212  
213 213  Both cases make it possible to:
214 214  
215 -* extract SDMX-ML files from the database that will be made available to be pulled by data collectors;
206 +* extract [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] files from the database that will be made available to be pulled by data collectors;
216 216  * allow the database to be queried directly through a web service.
217 217  
218 -Whichever type of data producer architecture is involved, a mapping process between structural metadata may be necessary, as explained in 5.3.
209 +Whichever type of data producer architecture is involved, a mapping process between [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] may be necessary, as explained in 5.3.
219 219  
220 220  == 8.8 Registry Interfaces ==
221 221  
... ... @@ -224,26 +224,26 @@
224 224  The Registry Interfaces are:
225 225  
226 226  * Notify Registry Event
227 -* Submit Subscription Request
228 -* Submit Subscription Response
218 +* Submit [[Subscription>>doc:sdmx:Glossary.Subscription.WebHome]] Request
219 +* Submit [[Subscription>>doc:sdmx:Glossary.Subscription.WebHome]] Response
229 229  * Submit Registration Request
230 230  * Submit Registration Response
231 231  * Query Registration Request
232 232  * Query Registration Response
233 -* Query Subscription Request
234 -* Query Subscription Response
224 +* Query [[Subscription>>doc:sdmx:Glossary.Subscription.WebHome]] Request
225 +* Query [[Subscription>>doc:sdmx:Glossary.Subscription.WebHome]] Response
235 235  * Submit Structure Request
236 236  * Submit Structure Response
237 237  
238 -Applications communicate with the Registry using either the SDMX-ML Registry Interface message (which contains an XML element that defines the actual interface) or individual messages – one for each interface.
229 +Applications communicate with the Registry using either the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] Registry Interface message (which contains an XML element that defines the actual interface) or individual messages – one for each interface.
239 239  
240 240  In more technical terms the Registry interfaces are invoked in one of two ways:
241 241  
242 -1. The interface is the name of the root node of the SDMX-ML document.
243 -1. The interface is invoked as a child element of the RegistryInterface message where the RegistryInterface is the root node of the SDMX-ML document.
233 +1. The interface is the name of the root node of the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] document.
234 +1. The interface is invoked as a child element of the RegistryInterface message where the RegistryInterface is the root node of the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] document.
244 244  
245 -In addition to these interfaces the Registry must support a mechanism for querying for structural metadata.
236 +In addition to these interfaces the Registry must support a mechanism for querying for [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]].
246 246  
247 -All these interactions with the Registry – with the exception of Notify Registry Event – are designed in pairs. The first document (the one which invokes the SDMX Registry Interface) is a “Request” document. The message returned by the interface is a “Response” document.
238 +All these interactions with the Registry – with the exception of Notify Registry Event – are designed in pairs. The first document (the one which invokes the [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] Interface) is a “Request” document. The message returned by the interface is a “Response” document.
248 248  
249 -It should be noted that all interactions are assumed to be synchronous, with the exception of Notify Registry Event. This document is sent by the SDMX-Registry to all subscribers whenever an even occurs to which any users have subscribed. Thus, it does not conform to the request-response pattern, because it is inherently asynchronous.
240 +It should be noted that all interactions are assumed to be synchronous, with the exception of Notify Registry Event. This document is sent by the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-Registry to all subscribers whenever an even occurs to which any users have subscribed. Thus, it does not conform to the request-response pattern, because it is inherently asynchronous.
SUZ.Methodology.Code.MethodologyClass[0]
index
... ... @@ -1,1 +1,0 @@
1 -9
SKMS.Methodology.Code.MethodologyClass[0]
Index
... ... @@ -1,0 +1,1 @@
1 +9