Skip to Content
Last modified by Helena on 2025/09/10 11:19
From version 3.1
edited by Helena
on 2025/05/23 22:19
Change comment: There is no comment for this version
To version 1.2
edited by Artur
on 2025/05/20 14:28
Change comment: Update document after refactoring.

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -xwiki:XWiki.helena
1 +xwiki:XWiki.arturkryazhev
Content
... ... @@ -4,49 +4,53 @@
4 4  
5 5  == 15.1 Scope ==
6 6  
7 -This annex includes a set of example queries which walk through the process of querying a [[data source>>doc:sdmx:Glossary.Data source.WebHome]] for the purposes of data discovery. It also includes an additional section which details some of the advanced features of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] query message.
7 +This annex includes a set of example queries which walk through the process of querying a data source for the purposes of data discovery. It also includes an additional section which details some of the advanced features of the SDMX query message.
8 8  
9 9  == 15.2 Discovering Categories ==
10 10  
11 -The first step a user may take in discovering data is to see the [[categories>>doc:sdmx:Glossary.Category.WebHome]] which data might be classified against. In this example scenario, a user is trying to find exchange rate data from the European Central Bank. The first step is to discover if there is a [[category>>doc:sdmx:Glossary.Category.WebHome]] for such data.
11 +The first step a user may take in discovering data is to see the categories which data might be classified against. In this example scenario, a user is trying to find exchange rate data from the European Central Bank. The first step is to discover if there is a category for such data.
12 12  
13 13  === 15.2.1 REST ===
14 14  
15 -In the REST syntax, there is no means to search based on text, so the simplest means for the user to find the exchange rate [[category>>doc:sdmx:Glossary.Category.WebHome]] is to retrieve all [[categories>>doc:sdmx:Glossary.Category.WebHome]] and filter through the results. Such a query would be structured as follows:
15 +In the REST syntax, there is no means to search based on text, so the simplest means for the user to find the exchange rate category is to retrieve all categories and filter through the results. Such a query would be structured as follows:
16 16  
17 17  http:~/~/ws-entry-point/categoryscheme
18 18  
19 19  A sample result for this can be seen in the sample file, ecb_query/ecb_exr_category_rest.xml.
20 20  
21 -Note that the entire [[category scheme>>doc:sdmx:Glossary.Category scheme.WebHome]] is returned. If the [[data source>>doc:sdmx:Glossary.Data source.WebHome]] had more [[category schemes>>doc:sdmx:Glossary.Category scheme.WebHome]], these would have been returned as well.
21 +Note that the entire category scheme is returned. If the data source had more category schemes, these would have been returned as well.
22 22  
23 23  === 15.2.2 SOAP ===
24 24  
25 -Using the SOAP query, one can query for a [[category>>doc:sdmx:Glossary.Category.WebHome]] by name, and request that only the matched items be returned. The sample file, ecb_query/ecb_exr_category_soap.xml demonstrates a query for [[category schemes>>doc:sdmx:Glossary.Category scheme.WebHome]] where a [[category>>doc:sdmx:Glossary.Category.WebHome]] contains the text "Exchange rate" in its English name. Note that it also request that only matched items and their children be returned. The result file for this query is ecb_query/ecb_exr_category_soap.xml. It can be seen from this result file that only the relevant [[categories>>doc:sdmx:Glossary.Category.WebHome]] are returned.
25 +Using the SOAP query, one can query for a category by name, and request that only the matched items be returned. The sample file, ecb_query/ecb_exr_category_soap.xml demonstrates a query for category schemes where a category contains the text "Exchange rate" in its English name. Note that it also request that only matched items
26 26  
27 +and their children be returned. The result file for this query is ecb_query/ecb_exr_category_soap.xml. It can be seen from this result file that only the relevant categories are returned.
28 +
27 27  == 15.3 Discovering Data Structures ==
28 28  
29 -After the user queried for [[categories>>doc:sdmx:Glossary.Category.WebHome]], he can now examine these [[categories>>doc:sdmx:Glossary.Category.WebHome]] in order to determine how to proceed with the data discovery process. Suppose that upon examination the user decide that he actually wanted data for the effective exchange rate [[category>>doc:sdmx:Glossary.Category.WebHome]] (2018773.2018810.2018779.2018795). The user will want to find [[data flows>>doc:sdmx:Glossary.Dataflow.WebHome]] which are categorised against this [[category>>doc:sdmx:Glossary.Category.WebHome]].
31 +After the user queried for categories, he can now examine these categories in order to determine how to proceed with the data discovery process. Suppose that upon examination the user decide that he actually wanted data for the effective exchange rate category (2018773.2018810.2018779.2018795). The user will want to find data flows which are categorised against this category.
30 30  
31 31  === 15.3.1 REST ===
32 32  
33 -In the REST syntax, the user will have to query for the [[category scheme>>doc:sdmx:Glossary.Category scheme.WebHome]] in which the exchange rate is defined and request that categorisations and their references be returned. This query would be as follows:
35 +In the REST syntax, the user will have to query for the category scheme in which the exchange rate is defined and request that categorisations and their references be returned. This query would be as follows:
34 34  
35 35  http:~/~/ws-entry-point/categoryscheme/ECB/SDW_ECONOMIC_CONCEPTS/1.0?references="parentan dsiblings"
36 36  
37 -Note the references parameter. It specifies parent (i.e. objects referencing the queried object) and siblings, objects referenced directly from the parent object. This will return categorisations which reference the [[categories>>doc:sdmx:Glossary.Category.WebHome]] of the queried [[category scheme>>doc:sdmx:Glossary.Category scheme.WebHome]] and the objects which these categorisations categorise. The sample file, ecb_query/ecb_exr_dataflow_rest.xml shows the result of this query. Note that this assumes that there is only one [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]] categorised against the one [[category>>doc:sdmx:Glossary.Category.WebHome]]. In reality, all categorisations and their objects for the entire [[category scheme>>doc:sdmx:Glossary.Category scheme.WebHome]] would be returned.
39 +Note the references parameter. It specifies parent (i.e. objects referencing the queried object) and siblings, objects referenced directly from the parent object. This will return categorisations which reference the categories of the queried category scheme and the objects which these categorisations categorise. The sample file,
38 38  
41 +ecb_query/ecb_exr_dataflow_rest.xml shows the result of this query. Note that this assumes that there is only one data flow categorised against the one category. In reality, all categorisations and their objects for the entire category scheme would be returned.
42 +
39 39  === 15.3.2 SOAP ===
40 40  
41 -The SOAP syntax is able to take a much more direct approach. It can query directly for categorisations which use the specific effective exchange rate [[category>>doc:sdmx:Glossary.Category.WebHome]], and only categorisation which categorise [[data flows>>doc:sdmx:Glossary.Dataflow.WebHome]]. Further, it can omit the actual categorisation from the results, leaving only the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] to be returned. The sample file, ecb_query\ecb_exr_dataflow_query.xml shows this sample query and ecb_query\ecb_exr_dataflow_soap.xml shows the result.
45 +The SOAP syntax is able to take a much more direct approach. It can query directly for categorisations which use the specific effective exchange rate category, and only categorisation which categorise data flows. Further, it can omit the actual categorisation from the results, leaving only the dataflow to be returned. The sample file, ecb_query\ecb_exr_dataflow_query.xml shows this sample query and ecb_query\ecb_exr_dataflow_soap.xml shows the result.
42 42  
43 43  == 15.4 Discovering Data ==
44 44  
45 -Now that the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] has been found for the effective exchange rate data, the actual data reported against this flow can be retrieved.
49 +Now that the dataflow has been found for the effective exchange rate data, the actual data reported against this flow can be retrieved.
46 46  
47 47  === 15.4.1 REST ===
48 48  
49 -In the REST syntax, the data query is very straight forward for querying for an entire [[data set>>doc:sdmx:Glossary.Data set.WebHome]] for a give [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]], in this case the effective exchange rates flow (2034482):
53 +In the REST syntax, the data query is very straight forward for querying for an entire data set for a give data flow, in this case the effective exchange rates flow (2034482):
50 50  
51 51  http:~/~/ws-entry-point/data/2034482
52 52  
... ... @@ -54,19 +54,19 @@
54 54  
55 55  === 15.4.2 SOAP ===
56 56  
57 -The query in the SOAP syntax is just as simple, as shown in ecb_query/ecb_exr_data_query.xml. The request is for structure specific data, but does not specify which [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] to orient the data on. The implication is that the [[data source>>doc:sdmx:Glossary.Data source.WebHome]] will return the data oriented with the [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]] it chooses at the observation (% style="color:#e74c3c" %)level(%%). In this case, it is the time [[dimension>>doc:sdmx:Glossary.Dimension.WebHome]].
61 +The query in the SOAP syntax is just as simple, as shown in ecb_query/ecb_exr_data_query.xml. The request is for structure specific data, but does not specify which dimension to orient the data on. The implication is that the data source will return the data oriented with the dimension it chooses at the observation level. In this case, it is the time dimension.
58 58  
59 59  The result for this query is the same as the REST and is shown in the sample file ecb_query\ecb_exr_data.xml.
60 60  
61 -Technically, the query for the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]] was not actually necessary to discover the data using the SOAP query. It is possible to query for data directly by a [[category>>doc:sdmx:Glossary.Category.WebHome]] - with the result being any data in which the [[data set>>doc:sdmx:Glossary.Data set.WebHome]], the data structure, the [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]], or the [[provision agreement>>doc:sdmx:Glossary.Provision agreement.WebHome]] which is categorised against the referenced [[category>>doc:sdmx:Glossary.Category.WebHome]].
65 +Technically, the query for the dataflow was not actually necessary to discover the data using the SOAP query. It is possible to query for data directly by a category - with the result being any data in which the data set, the data structure, the data flow, or the provision agreement which is categorised against the referenced category.
62 62  
63 63  == 15.5 Better Data Queries ==
64 64  
65 -The above example assumes a very simple scenario, where the data itself was not actually queried. In actuality, such a query may not be wise as the volume of data returned could be quite large. In a more realistic scenario, one might choose to first examine the data structure for the [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]] in order to determine what data might be available. After the full data structure details are returned, one can examine the dimensionality of the data and even look at the available [[codes>>doc:sdmx:Glossary.Code.WebHome]] to do things such as search for exchange rates for only select [[currencies>>doc:sdmx:Glossary.Currency.WebHome]].
69 +The above example assumes a very simple scenario, where the data itself was not actually queried. In actuality, such a query may not be wise as the volume of data returned could be quite large. In a more realistic scenario, one might choose to first examine the data structure for the data flow in order to determine what data might be available. After the full data structure details are returned, one can examine the dimensionality of the data and even look at the available codes to do things such as search for exchange rates for only select currencies.
66 66  
67 67  === 15.5.1 REST ===
68 68  
69 -To retrieve the full details of the data structure in REST based on the [[dataflow>>doc:sdmx:Glossary.Dataflow.WebHome]], one simply queries for the [[data flow>>doc:sdmx:Glossary.Dataflow.WebHome]] and all descendant references:
73 +To retrieve the full details of the data structure in REST based on the dataflow, one simply queries for the data flow and all descendant references:
70 70  
71 71  http:~/~/ws-entry-point/dataflow/ECB/2034482/1.0?references="descendants"
72 72  
... ... @@ -78,7 +78,7 @@
78 78  
79 79  == 15.6 Other Structural Metadata Query Features ==
80 80  
81 -There is a sample set included with this document that serve to highlight some of the more advanced mechanisms in the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] queries. These sample files are as follows.
85 +There is a sample set included with this document that serve to highlight some of the more advanced mechanisms in the structural metadata queries. These sample files are as follows.
82 82  
83 83  Directory Name: query
84 84