Wiki source code of 10 Community Management

Last modified by Artur on 2025/09/10 11:19

version	line-number	content
1.1	1	{{box title="Contents"}}
	2	{{toc/}}
	3	{{/box}}
	4
	5	== 10.1 Scope of this Chapter ==
	6
2.1	7	Many organizations have a community of data reporters, data sharing agencies, or [[data consumers>>doc:sdmx:Glossary.Data consumer.WebHome]]. This places responsibilities upon the organization in terms of publishing and dissemination of [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]], setting up [[hubs>>doc:sdmx:Glossary.Hub (dissemination architecture).WebHome]] for data sharing, administration of [[maintenance agencies>>doc:sdmx:Glossary.Maintenance agency.WebHome]], usernames etc.
1.1	8
	9	These aspects are dealt with in this Chapter.
	10
	11	== 10.2 Maintenance Agency Maintenance ==
	12
2.1	13	All [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is owned and maintained by a [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]] (Agency identified by agencyID in the schemas). It is vital to the integrity of the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] that there are no conflicts in agencyID.
1.1	14
4.2	15	The maintenance of [[maintenance agencies>>doc:sdmx:Glossary.Maintenance agency.WebHome]] in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is a devolved function. Any organization registered as a [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]] can itself set up and maintain its own [[maintenance agencies>>doc:sdmx:Glossary.Maintenance agency.WebHome]]. However, in order for this devolved system to work there must be a “top-level” [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]] list (called an [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]) which is itself maintained by the recognized top-level Agency. This Agency is [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]. Any organization registered in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] can itself maintain its own [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] of sub-agencies. Only one such [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] is allowed for any recognized Agency. With the exception of the “[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]” Agency, a recognized Agency must itself be registered in a “parent” [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]].
1.1	16
2.1	17	The following rules are a more formal definition of the way the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] agency system works:
1.1	18
2.1	19	1. Agencies are maintained in an [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]].
	20	1. The [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]] of the [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] must be registered as an Agency in a (different) [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] (the “parent” [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]]).
4.2	21	1. The “top-level” agency is [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] and this [[agency scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] is maintained by [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].
	22	1. Agencies registered in the top-level scheme can themselves maintain a single [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]]. [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is an agency in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[agency scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]], thus allowing [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] be a [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]]. Agencies in this (child) scheme can themselves maintain a single [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] and so on.
	23	1. The [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] cannot be versioned and so takes a default version number of 1.0, and it cannot be made “final”.
2.1	24	1. There can be only one [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] maintained by any one Agency. It has a fixed Id of AGENCIES.
4.2	25	1. The format of the agency identifier is agencyId.agencyID etc. The top-level agency in this identification mechanism is the agency registered in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] [[agency scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]]. In other words, [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is not a part of the hierarchical ID structure for agencies.
2.1	26	1. [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is, itself, a [[maintenance agency>>doc:sdmx:Glossary.Maintenance agency.WebHome]].
1.1	27
	28	This supports a hierarchical structure of agencyID.
	29
	30	An example is shown below.
	31
2.1	32	[[image:SDMX_2-1_User_Guide_draft_0-1_html_cf0235008d0d8c4c.jpg\|\|data-xwiki-image-style-alignment="center" height="294" width="373"]]
1.1	33
	34	(% style="text-align: center;" %)
	35	{{id name="image_29"/}}Figure 29:Example of Hierarchic Structure of Agencies
	36
2.1	37	Each agency is identified by its full [[hierarchy>>doc:sdmx:Glossary.Hierarchy.WebHome]] excluding [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].
1.1	38
	39	The XML representing this structure is shown below.
	40
2.1	41	[[image:SDMX_2-1_User_Guide_draft_0-1_html_8f57e83ccb4d76c1.jpg\|\|data-xwiki-image-style-alignment="center" height="634" width="411"]]
1.1	42
	43	(% style="text-align: center;" %)
	44	{{id name="image_30"/}}Figure 30: Example Agency Schemes Showing a Hierarchy
	45
	46	__Example of Structure Definitions:__
	47
2.1	48	[[image:SDMX_2-1_User_Guide_draft_0-1_html_ced0c4e613850a25.jpg\|\|data-xwiki-image-style-alignment="center" height="251" width="433"]]
1.1	49
	50	(% style="text-align: center;" %)
	51	{{id name="image_31"/}}Figure 31: Example Showing Use of Agency Identifiers
	52
2.1	53	Each of these [[maintenance agencies>>doc:sdmx:Glossary.Maintenance agency.WebHome]] has an identical Codelist with the Id CL_BOP. However, each is uniquely identified by means of the hierarchic agency structure.
1.1	54
2.1	55	Clearly, in order for such a system to work there must be a mechanism that enables a user or organisation to discover the full “list” of [[maintenance agencies>>doc:sdmx:Glossary.Maintenance agency.WebHome]] or at least the [[Agency Scheme>>doc:sdmx:Glossary.Agency scheme.WebHome]] in which an organisation is registered. In order for this to be possible all [[Agency Schemes>>doc:sdmx:Glossary.Agency scheme.WebHome]] must be made known to a Global [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] and either maintained in that Registry or maintained and made available from a metadata source that is referenced from an entry in the [[Global Registry>>doc:sdmx:Glossary.Global registry.WebHome]]. For example, the [[Agency Schemes>>doc:sdmx:Glossary.Agency scheme.WebHome]] shown in Figure 30 could be “registered” in the [[Global Registry>>doc:sdmx:Glossary.Global registry.WebHome]] as follows:
1.1	56
2.1	57	[[image:SDMX_2-1_User_Guide_draft_0-1_html_348aea09d640ec2d.jpg\|\|data-xwiki-image-style-alignment="center" height="215" width="553"]]
1.1	58
	59	(% style="text-align: center;" %)
	60	{{id name="image_32"/}}Figure 32: Example XML Showing External References
	61
	62	== 10.3 Dissemination of Structural Metadata ==
	63
2.1	64	When [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] are disseminated or exchanged it is important that any consuming application has access to all of the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] that is referenced from structures such as a [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] or [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]] (i.e. in these cases the [[Concept Schemes>>doc:sdmx:Glossary.Concept scheme.WebHome]], [[Code Lists>>doc:sdmx:Glossary.Code list.WebHome]], [[Category Schemes>>doc:sdmx:Glossary.Category scheme.WebHome]] etc. that are “used” in the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] or the [[MSD>>doc:sdmx:Glossary.Metadata structure definition.WebHome]]). These structures can be embedded as complete structures in the file that is disseminated, or they can be embedded as “stubs” that contain the reference from where the structures can be retrieved. This referencing mechanism is achieved using one of the [[SDMX-ML>>doc:sdmx:Glossary.SDMX-ML.WebHome]] [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]]:
1.1	65
	66	* structureURL
	67	* serviceURL
	68
4.2	69	These [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] are available on every structure that is maintained e.g. it is at the level of the “maintained object” such as [[Code List>>doc:sdmx:Glossary.Code list.WebHome]], [[Data Structure Definition>>doc:sdmx:Glossary.Data structure definition.WebHome]]. In addition to the use of these [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] the [[attribute>>doc:sdmx:Glossary.Attribute.WebHome]] [[isExternalReference>>doc:sdmx:Glossary.isExternalReference.WebHome]] should be set to “true” indicating that the full definition of the structure is not available and must be retrieved from one of the reference [[attributes>>doc:sdmx:Glossary.Attribute.WebHome]] structureURL or serviceURL.
1.1	70
	71	It is therefore the responsibility of the agency maintaining these structures that the structures can be accessed using the URL. If the URL cannot be used to retrieve the structure then it is possible that applications using the structures will not be able to process the information properly. It follows that the content of structureURL and serviceURL should be deemed to be resolvable in the long term.
	72
2.1	73	The use of or a link to a shared community [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] is recommended for organizations that wish to store such structures. For example, the shared community registry can be used as the repository for the referenced structures.
1.1	74
2.1	75	Note that any [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] submitted to an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] (SubmitStructureRequest) must have resolvable references to all of the structures cross referenced in the submitted structure. If this is not the case then the submission may be rejected.
1.1	76
	77	== 10.4 Maintenance of Community Concept Roles ==
	78
	79	=== 10.4.1 Overview ===
	80
2.1	81	The [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] [[Components>>doc:sdmx:Glossary.Component.WebHome]] of [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] and [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]] can play a specific role in the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] and it is important to some applications that this role is specified. For instance, the following roles are some examples:
1.1	82
2.1	83	Frequency – in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] the content of this [[Component>>doc:sdmx:Glossary.Component.WebHome]] contains information on the frequency of the observation values
1.1	84
2.1	85	Geography - in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] the content of this [[Component>>doc:sdmx:Glossary.Component.WebHome]] contains information on the geographic location of the observation values
1.1	86
2.1	87	Unit of [[Measure>>doc:sdmx:Glossary.Measure.WebHome]] - in a [[data set>>doc:sdmx:Glossary.Data set.WebHome]] the content of this [[Component>>doc:sdmx:Glossary.Component.WebHome]] contains information on the [[unit of measure>>doc:sdmx:Glossary.Unit of measure.WebHome]] of the observation values
1.1	88
4.2	89	In order for these roles to be extensible and also to enable user communities to maintain community-specific roles, the roles are maintained in a controlled vocabulary which is implemented in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] as (% style="color:#e74c3c" %)[[Concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) in a [[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]].
1.1	90
4.2	91	It is possible to specify zero or more (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) roles for a [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], [[Measure>>doc:sdmx:Glossary.Measure.WebHome]] [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]] and Data [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]] (but not the ReportingYearStartDay). The Time [[Dimension>>doc:sdmx:Glossary.Dimension.WebHome]], Primary [[Measure>>doc:sdmx:Glossary.Measure.WebHome]], and the [[Attribute>>doc:sdmx:Glossary.Attribute.WebHome]] ReportingYearStartDay have explicitly defined roles and cannot be further specified with additional (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) roles.
1.1	92
	93	=== 10.4.2 Maintaining and Using Concept Roles ===
	94
4.2	95	The mechanism for maintaining and using (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) roles is described in the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Standards Section 6: Technical Notes. It is the responsibility of Agencies to ensure their community knows which (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) in which [[concept schemes>>doc:sdmx:Glossary.Concept scheme.WebHome]] play a “role” and the significance and interpretation of this role. In other words, such (% style="color:#e74c3c" %)[[concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) must be known by applications, there is no technical mechanism that can inform an application on how to process such a “role”.
1.1	96
4.2	97	Clearly, Agencies defining DSDs can use (% style="color:#e74c3c" %)[[Concepts>>doc:sdmx:Glossary.Concept.WebHome]](%%) from any [[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]], even a [[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] not maintained by this Agency. It is therefore important that any [[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] that is referenced from the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]] must be available from a known URL to any application that needs to process the [[DSD>>doc:sdmx:Glossary.Data structure definition.WebHome]], or be embedded in the file of [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] (this is no different from any other cross referenced structure as described in 10.3 above). Therefore, if an Agency decides to maintain its own [[Concept Scheme>>doc:sdmx:Glossary.Concept scheme.WebHome]] of (% style="color:#e74c3c" %)[[concept>>doc:sdmx:Glossary.Concept.WebHome]](%%) roles it needs to consider whether this scheme is “public” (i.e. it is required in the DSDs disseminated by the organisation) and, if so, ensure that it is made available.
1.1	98
	99	== 10.5 Hosting of a shared registry ==
	100
2.1	101	It is clear from the responsibilities of an Agency that there is a need to ensure [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] maintained by that Agency, and which is to be used outside of the organization that is the Agency, is made available. As more and more organizations use [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] and as more Agencies are created, it is clear that a central repository of [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] will benefit the “community” of the Agency. Many of these communities will exist already, as they will pre-date the existence of. The [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] standard formalizes the way a community exchanges and shares data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]], and as more [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] communities are created then the greater is the need for control over the maintenance and sharing of the [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]].
1.1	102
2.1	103	Community “Agencies” should consider taking on the role of hosting a shared [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] for the community. Clearly, there will be many of these shared registries and it may be necessary that these registries can be linked in a “federation” so that common [[structural metadata>>doc:sdmx:Glossary.Structural metadata.WebHome]] is accessible. This is especially true of the [[Agency Schemes>>doc:sdmx:Glossary.Agency scheme.WebHome]] that will exist: it will only be possible to validate an agencyID if the Agency is known and this may involve knowledge of the full [[hierarchy>>doc:sdmx:Glossary.Hierarchy.WebHome]] of Agencies which will be need to be built from the Agencies maintained in separate [[Agency Schemes>>doc:sdmx:Glossary.Agency scheme.WebHome]]. Furthermore, it is probable that many [[Code Lists>>doc:sdmx:Glossary.Code list.WebHome]] and [[Concept Schemes>>doc:sdmx:Glossary.Concept scheme.WebHome]] will not be disseminated with DSDs and MSDs, but rather be referenced from the structure file disseminated. This type of federated architecture is best supported by an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]], as this is the one of the key roles of an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]].
1.1	104
	105	== 10.6 Data Provider Maintenance ==
	106
2.1	107	An organization that collects data or [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] from other organizations (these are known as [[Data Providers>>doc:sdmx:Glossary.Data provider.WebHome]] in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]) must maintain a DataProvider Scheme. This is self evident as the collecting organization must know from which reporting organization data or [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] is received. Clearly, an organization can collect data or [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] in an [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] format without having an [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] DataProvider Scheme. Nevertheless, data collecting organizations which adopt [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] are encouraged to maintain an [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] DataProvider Scheme, or to be able to disseminate such a scheme from their own internal scheme.
1.1	108
2.1	109	If the collecting organization wishes to adopt the “pull” method of data/metadata reporting, or wishes to host an [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] where its community can publish the existence of data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] sources (by means of a data/metadata Registration), then it is mandatory to have a DataProvider Scheme. The relationship between the fundamental structures supporting the “pull” mechanism and data/metadata discovery is shown in the diagram below.
1.1	110
2.1	111	[[image:SDMX_2-1_User_Guide_draft_0-1_html_1ac2660026676010.jpg\|\|data-xwiki-image-style-alignment="center" height="512" width="575"]]
1.1	112
	113	(% style="text-align: center;" %)
	114	{{id name="image_33"/}}Figure 33: SDMX structures required for the “pull” reporting mechanism or for data/metadata discovery