Wiki source code of Guidelines on coding time transformations in SDMX

Last modified by Artur K. on 2026/05/29 14:28

version	line-number	content
1.1	1	{{box title="Contents"}}
	2	{{toc/}}
	3	{{/box}}
	4
	5	= 1. Problem Statement =
	6
1.6	7	Time transformation is defined as a time-related operation performed on a time series, solely involving observations of that time series. Examples of such time transformations are growth rates, cumulative sums over several periods and moving averages.
1.1	8
1.6	9	To express a time transformation, three elements are required: the type of transformation, the number of periods involved and the length of each period. Even though in theory you could express the base value and the transformation applied, it is much more practical, and in many cases sufficient in statistical data exchange, to transmit the time-transformed values themselves.
1.1	10
1.6	11	The operation to be coded can be expressed generically as such: For value //V// the transformation //T// was applied over //P// periods with frequency //F//.
1.1	12
	13	Examples:
	14
2.6	15	\|(% style="width:501px" %)Statement\|(% style="width:303px" %)T\|(% style="width:472px" %)P\|(% style="width:454px" %)F
	16	\|(% style="width:501px" %)Quarter on quarter growth rate\|(% style="width:303px" %)Growth rate\|(% style="width:472px" %)2\|(% style="width:454px" %)Q
	17	\|(% style="width:501px" %)Contribution to growth over 1 year (quarterly data)\|(% style="width:303px" %)Contribution to growth\|(% style="width:472px" %)4\|(% style="width:454px" %)Q
	18	\|(% style="width:501px" %)Contribution to growth over 1 year (annual data)\|(% style="width:303px" %)Contribution to growth\|(% style="width:472px" %)1\|(% style="width:454px" %)A
	19	\|(% style="width:501px" %)3 months moving average\|(% style="width:303px" %)Moving average\|(% style="width:472px" %)3\|(% style="width:454px" %)M
	20	\|(% style="width:501px" %)Annual index (reference year=100){{footnote}}Note that for the case of an index, it is useful to specify the reference base period in an additional attribute (see concept BASE_PER specified in the SDMX Glossary).{{/footnote}}\|(% style="width:303px" %)Index\|(% style="width:472px" %)1\|(% style="width:454px" %)A
1.1	21
	22	This guideline describes two methods that may be used to code a time transformation:
	23
1.6	24	1. A normalised, multi-concept approach that is described in section SDMX CONCEPTS FOR TIME TRANSFORMATIONS. The overall time span involved in the time transformation depends on the number of periods stated and the frequency of a series.
	25	1. A denormalised, compound concept approach that is described in section COMPOUND CODING FOR TIME TRANSFORMATIONS. The overall time span involved in the time transformation does not necessarily depend on the number of periods stated and the frequency of a series.
1.1	26
1.6	27	Both of these methods are included as separate use cases as served by each method. The aim of this document is to demonstrate that guidance and a standard approach is available and promoted for each use case. The use cases are described in the related sections.
1.1	28
2.3	29	Further recommended code values for expressing general statistical concepts such as "not applicable", etc., can be found in section “Generic codes” of the "Guidelines for the creation and management of SDMX Cross-Domain Code Lists" (to be found under “Guidelines” on the official SDMX website{{footnote}}http://sdmx.org/{{/footnote}}).
1.1	30
1.2	31	= 2. SDMX Concepts for Time Transformations =
1.1	32
1.6	33	SDMX defines two cross domain concepts for the purpose of coding time transformations: Time transformation type (ID TIMETRANS_TYPE) and time transformation periods (ID TIMETRANS_PER). The concept TIMETRANS_TYPE is coded with a cross domain code list. The concept TIMETRANS_PER is coded with a coded list of integers.
1.1	34
1.2	35	== 2.1 Time Transformation Type ==
1.1	36
	37	Definition: This concept provides coded information about time-related transformation types of time series.
	38
	39	Concept ID: The concept ID is TIMETRANS_TYPE.
	40	Code List Name: Code list for Time Transformation Type.
	41	Code List ID: CL_TIMETRANS_TYPE.
	42	Established international standard(s) used as input for the code list: None.
	43
	44	Version: 1.0, 15 September 2016
	45
2.7	46	\|Recommended code value\|(% style="width:309px" %)Recommended code description\|(% style="width:1262px" %)Annotation
	47	\|N\|(% style="width:309px" %)Non transformed\|(% style="width:1262px" %)TIMETRANS_PER is always 1, since a non-transformed number covers by definition a single period
	48	\|A\|(% style="width:309px" %)Average\|(% style="width:1262px" %)Moving average, i.e. an operation that preserves the frequency of the series
	49	\|C\|(% style="width:309px" %)Cumulated sum\|(% style="width:1262px" %)
	50	\|D\|(% style="width:309px" %)Difference\|(% style="width:1262px" %)
	51	\|DD\|(% style="width:309px" %)Difference, second order\|(% style="width:1262px" %)A second order difference is the delta of deltas
	52	\|F\|(% style="width:309px" %)Growth rate, flow over stock\|(% style="width:1262px" %)
	53	\|FC\|(% style="width:309px" %)Contribution to growth, flow over stock\|(% style="width:1262px" %)
	54	\|G\|(% style="width:309px" %)Growth rate\|(% style="width:1262px" %)
	55	\|GC\|(% style="width:309px" %)Contribution to growth\|(% style="width:1262px" %)
	56	\|I\|(% style="width:309px" %)Index\|(% style="width:1262px" %)In the usual case, the index is fixed to 100 for a specific reference period, in most cases a year. It is recommended that the DSD contains an additional attribute BASE_PER (type ObservationalTimePeriod), which specifies the reference period of the index. In special cases (e.g. National Accounts chain linking), the index is fixed to a value different to 100 in the reference year. In these cases the BASE_PER attribute is even more important.
	57	\|LA\|(% style="width:309px" %)Annualised levels\|(% style="width:1262px" %)This relates to stock versus flow series. For example, many countries publish their Quarterly National Accounts (QNA) at quarterly level, which means that annual Gross Domestic Product (GDP) is the sum of the four quarters, whereas some countries publish their QNA at annual level (e.g. US), which means that annual GDP is the average of the four quarters. In order to present quarterly data in comparable levels across countries and to derive zone aggregates, quarterly data expressed at quarterly levels are “transformed” to annual levels (i.e. multiplied by four) and have this code.
	58	\|S\|(% style="width:309px" %)Shifted\|(% style="width:1262px" %)The time series was moved back or forth in time. This may for instance be used when non-calendar year series are aligned to the calendar year using certain estimation formulas.
	59	\|_O\|(% style="width:309px" %)Other transformation\|(% style="width:1262px" %)This code is taken from the guidelines on generic codes, specifying "Other". In that context it should be used if more complex transformations are applied. An explanation of the transformation or a transformation script should be given in a comment field.
1.1	60
1.3	61	== 2.2 Time Transformation Periods ==
1.1	62
	63	Definition: This concept provides information about the number of periods used for a time-related transformation of the time series.
	64
	65	Concept ID: The concept ID is TIMETRANS_PER.
	66	Code List Name: Code list for Time Transformation Periods.
	67	Code List ID: CL_TIMETRANS_PER.
	68	Established international standard(s) used as input for the code list: None.
	69
	70	Version: 1.0, 15 September 2016
	71
2.7	72	(% style="width:1172.96px" %)
	73	\|(% style="width:320px" %)Recommended code value\|(% style="width:408px" %)**Recommended
	74	code description\|(% style="width:442px" %)Annotation**
	75	\|(% style="width:320px" %)1\|(% style="width:408px" %)One\|(% style="width:442px" %)
	76	\|(% style="width:320px" %)2\|(% style="width:408px" %)Two\|(% style="width:442px" %)
	77	\|(% style="width:320px" %)etc.\|(% style="width:408px" %)etc.\|(% style="width:442px" %)
1.1	78
1.4	79	== 2.3 Relation of transformation coding to transformation rules ==
1.1	80
1.6	81	Transformation can also be expressed with transformation rules using a syntax such as the Validation and Transformation Language (VTL). Following the transformation graph model behind VTL, the transformation coding suggested in this guideline can be seen complementary with using transformation rules in VTL. The idea is that a coded non-transformed time series is transformed using a VTL rule and the result is then coded again with transformation codes for further data exchange. This principle is shown in the graph below:
1.1	82
2.7	83	[[image:1769510498202-796.png]]
1.1	84
1.6	85	Using the two concepts as suggested above for coding the type of transformation applied and the number of periods covered will additionally ensure that the parameters used for the formula are directly used in the coding of the resulting series. Thus no complex mapping is needed. The transformation applied is linked to the transformation type concept and the number of periods used for the calculation is linked to the transformation periods concept.
1.1	86
	87	Example:
	88
2.3	89	The formula for a simple annual growth rate can be expressed as follows:{{footnote}}Note: often growth rates are expressed as percentage growth, in which case the value is multiplied with 100%. This is however not relevant for this guideline and is left out for simplicity.{{/footnote}}
1.1	90
2.8	91	[[image:1769510529532-579.png]]
1.1	92
2.9	93	A growth rate over P years in year T is the difference between the current year value and the value P years ago related to the value P years ago; with G being the growth rate, V being the absolute value, T being the time (year) and P the number of periods.
1.1	94
1.6	95	The growth rate formula can be expressed in VTL and linked to transformation type G. The year T is linked to the respective year in the time series and the parameter P is linked to the transformation period concept.
1.1	96
	97	Example:
	98
2.12	99	\|(% style="width:427px" %)Year →\|(% style="width:230px" %)2010\|(% style="width:365px" %)2011\|(% style="width:386px" %)2012\|(% style="width:322px" %)2013
2.9	100	\|(% style="width:427px" %)GDP Level\|(% style="width:230px" %)500\|(% style="width:365px" %)505\|(% style="width:386px" %)510\|(% style="width:322px" %)505
	101	\|(% style="width:427px" %)**Growth rate,
	102	period on period**\|(% style="width:230px" %) \|(% style="width:365px" %)0.0100\|(% style="width:386px" %)0.0099\|(% style="width:322px" %)-0.0098
	103	\|(% style="width:427px" %)Formula\|(% style="width:230px" %) \|(% style="width:365px" %)[[image:1769510625588-573.png]]\|(% style="width:386px" %)[[image:1769510654674-828.png]]\|(% style="width:322px" %)[[image:1769510676535-250.png]]
	104	\|(% style="width:427px" %)**Growth rate,
	105	over 2 periods**\|(% style="width:230px" %) \|(% style="width:365px" %) \|(% style="width:386px" %)0.0200\|(% style="width:322px" %)0.0000
	106	\|(% style="width:427px" %)Formula\|(% style="width:230px" %) \|(% style="width:365px" %) \|(% style="width:386px" %)[[image:1769510701118-480.png]]\|(% style="width:322px" %)[[image:1769510718908-350.png]]
1.1	107
	108	When looking at the formulas, you can see that the same parameters that are used to call a transformation service can be used to code the resulting series, which makes it very easy for data processing systems to ensure consistency between calculations and coding of results:
	109
2.12	110	\|Year →\|2011\|(% rowspan="4" %)(((
1.1	111	Transformed series:
	112
2.10	113	REF_YEAR → 2011
1.1	114
2.10	115	OBS_VALUE → 0.0100
1.1	116
2.10	117	TRANS_TYPE → G (Formula / VTL function)
1.1	118
2.10	119	TRANS_PER → 1
1.1	120	)))
	121	\|GDP Level\|505
	122	\|**Growth rate,
	123	period on period**\|0.0100
2.10	124	\|Formula\|[[image:1769510755278-324.png]]
1.1	125
1.6	126	This is especially useful when only transformed series should be exchanged and level series or transformations are not subject to exchange. An example could be GDP growth rates, where for early estimates often level series are still under embargo, whereas growth rates are publishable.
1.1	127
1.4	128	== 2.4 Recommendation ==
1.1	129
	130	Where possible, it is recommended to use the above solution with the two concepts TIMETRANS_TYPE and TIMETRANS_PER to express time transformations because:
	131
	132	* this method separates the type of transformation and the number of periods involved, therefore the coding of time transformation is simpler with no redundancy;
	133	* it is possible to add extra concepts if required without introducing ambiguity;
	134	* the coded transformations can be linked directly with transformation formulas.
	135
1.4	136	= 3. Compound coding for time transformations =
1.1	137
1.4	138	== 3.1 Known Limitations ==
	139
1.6	140	The normalised approach as presented above does not support the definition of mixed-frequency time transformations – like monthly series of annual growth rates – since there is only a single frequency dimension available. This also means that when annual growth rates are expressed in a quarterly dataset, the time transformation period would need to be modified (i.e. when frequency changes from A to Q, the number of periods need to be quadrupled).
1.1	141
	142	A "transformation frequency" might be added to keep the normalised approach also for those cases.
	143
1.6	144	It also does not allow to directly code complex transformations, like transforming already transformed series (like the period-on-period growth rate of a four-period cumulative sum). For that case it is recommended to use the generic code "_O - Other" to specific that another transformation has been applied and provide the explanation or the transformation script in a comment field.
1.1	145
1.6	146	However, both of these use cases may lead to a quite complex data structures or issues if various different complex transformations should be coded. Thus an alternative solution is presented in chapter 3 for cases where these use cases need to be covered and additional concepts should not be added to the data structure.
1.1	147
	148	In case the mixed frequencies or complex transformations as outlined above are needed in a simpler way and normalisation does not need to be strictly enforced, a composite code list CL_TIMETRANS may be created.
	149
2.4	150	The number of periods in the code follows the frequency of the series unless stated otherwise. Example: code G3Y refers to a three-year growth rate, irrespective of the series frequency. For complex transformations, the codes that would be used for the respective transformations can be concatenated and separated by an underscore{{footnote}}Example:
	151	G1_C4 Growth rate, period on period, over 4-period cumulated sum{{/footnote}}.
1.1	152
	153	Example for composite CL_TIMETRANS:
	154
2.13	155	\|(% style="width:227px" %)Recommended code value\|(% style="width:527px" %)Recommended code description\|(% style="width:976px" %)Annotation
	156	\|(% style="width:227px" %)N\|(% style="width:527px" %)Non transformed data\|(% style="width:976px" %)
	157	\|(% style="width:227px" %)A2\|(% style="width:527px" %)2-period moving average\|(% style="width:976px" %)Period on period
	158	\|(% style="width:227px" %)A3\|(% style="width:527px" %)3-period moving average\|(% style="width:976px" %)
	159	\|(% style="width:227px" %)A4\|(% style="width:527px" %)4-period moving average\|(% style="width:976px" %)
	160	\|(% style="width:227px" %)A6\|(% style="width:527px" %)6-period moving average\|(% style="width:976px" %)
	161	\|(% style="width:227px" %)A12\|(% style="width:527px" %)12-period moving average\|(% style="width:976px" %)
	162	\|(% style="width:227px" %)C3\|(% style="width:527px" %)3-period cumulated sum\|(% style="width:976px" %)
	163	\|(% style="width:227px" %)C4\|(% style="width:527px" %)4-period cumulated sum\|(% style="width:976px" %)
	164	\|(% style="width:227px" %)C6\|(% style="width:527px" %)6-period cumulated sum\|(% style="width:976px" %)
	165	\|(% style="width:227px" %)C12\|(% style="width:527px" %)12-period cumulated sum\|(% style="width:976px" %)
	166	\|(% style="width:227px" %)C16\|(% style="width:527px" %)16-period cumulated sum\|(% style="width:976px" %)
	167	\|(% style="width:227px" %)D2\|(% style="width:527px" %)Differences, period on period, first order\|(% style="width:976px" %)
	168	\|(% style="width:227px" %)DD\|(% style="width:527px" %)Differences, period on period, second order\|(% style="width:976px" %)
	169	\|(% style="width:227px" %)D4\|(% style="width:527px" %)Difference, period on 4 periods, first order\|(% style="width:976px" %)
	170	\|(% style="width:227px" %)F2\|(% style="width:527px" %)Growth rate, flow over stock, over two periods\|(% style="width:976px" %)Period on period
	171	\|(% style="width:227px" %)F3\|(% style="width:527px" %)Growth rate, flow over stock ,over 3 periods\|(% style="width:976px" %)
	172	\|(% style="width:227px" %)F4\|(% style="width:527px" %)Growth rate, flow over stock over 4 periods\|(% style="width:976px" %)
	173	\|(% style="width:227px" %)F6\|(% style="width:527px" %)Growth rate, flow over stock over 6 periods\|(% style="width:976px" %)
	174	\|(% style="width:227px" %)F12\|(% style="width:527px" %)Growth rate, flow over stock over 12 periods\|(% style="width:976px" %)
	175	\|(% style="width:227px" %)FO2\|(% style="width:527px" %)Contribution to growth rate, flow over stock, over two periods\|(% style="width:976px" %)Period on period
	176	\|(% style="width:227px" %)FO3\|(% style="width:527px" %)Contribution to growth rate, flow over stock, over 3 periods\|(% style="width:976px" %)
	177	\|(% style="width:227px" %)FO4\|(% style="width:527px" %)Contribution to growth rate, flow over stock, over 4 periods\|(% style="width:976px" %)
	178	\|(% style="width:227px" %)FO6\|(% style="width:527px" %)Contribution to growth rate, flow over stock, over 6 periods\|(% style="width:976px" %)
	179	\|(% style="width:227px" %)FO12\|(% style="width:527px" %)Contribution to growth rate, flow over stock, over 12 periods\|(% style="width:976px" %)
	180	\|(% style="width:227px" %)FO16\|(% style="width:527px" %)Contribution to growth rate, flow over stock, over 16 periods\|(% style="width:976px" %)
	181	\|(% style="width:227px" %)G2\|(% style="width:527px" %)Growth rate, over two periods\|(% style="width:976px" %)Period on period
	182	\|(% style="width:227px" %)G3\|(% style="width:527px" %)Growth rate over 3 periods\|(% style="width:976px" %)
	183	\|(% style="width:227px" %)G4\|(% style="width:527px" %)Growth rate over 4 periods\|(% style="width:976px" %)
	184	\|(% style="width:227px" %)G6\|(% style="width:527px" %)Growth rate over 6 periods\|(% style="width:976px" %)
	185	\|(% style="width:227px" %)G10\|(% style="width:527px" %)Growth rate, over 10 periods\|(% style="width:976px" %)
	186	\|(% style="width:227px" %)G12\|(% style="width:527px" %)Growth rate over 12 periods\|(% style="width:976px" %)
	187	\|(% style="width:227px" %)GR\|(% style="width:527px" %)Growth rate, over reference year\|(% style="width:976px" %)
	188	\|(% style="width:227px" %)GO2\|(% style="width:527px" %)Contribution to growth rate, over 2 periods\|(% style="width:976px" %)Period on period
	189	\|(% style="width:227px" %)GO3\|(% style="width:527px" %)Contribution to growth rate, over 3 periods\|(% style="width:976px" %)
	190	\|(% style="width:227px" %)GO4\|(% style="width:527px" %)Contribution to growth rate, over 4 periods\|(% style="width:976px" %)
	191	\|(% style="width:227px" %)GO6\|(% style="width:527px" %)Contribution to growth rate, over 6 periods\|(% style="width:976px" %)
	192	\|(% style="width:227px" %)GO12\|(% style="width:527px" %)Contribution to growth rate, over 12 periods\|(% style="width:976px" %)
	193	\|(% style="width:227px" %)LA\|(% style="width:527px" %)Annualised levels\|(% style="width:976px" %)This relates to stock versus flow series. For example, many countries publish their QNA at quarterly level, which means that annual GDP is the sum of the four quarters, whereas some countries publish their QNA at annual level (e.g. US), which means that annual GDP is the average of the four quarters. In order to present quarterly data in comparable levels across countries and to derive zone aggregates, quarterly data expressed at quarterly levels are “transformed” to annual levels (i.e. multiplied by four) and have this code.
	194	\|(% style="width:227px" %)G1Y\|(% style="width:527px" %)Growth rate, over 1 year\|(% style="width:976px" %)
	195	\|(% style="width:227px" %)F1Y\|(% style="width:527px" %)Growth rate, flow over stock, over 1 year\|(% style="width:976px" %)
	196	\|(% style="width:227px" %)D1Y\|(% style="width:527px" %)Difference, over 1 year\|(% style="width:976px" %)
	197	\|(% style="width:227px" %)G3Y\|(% style="width:527px" %)Growth rate, over 3 years\|(% style="width:976px" %)
	198	\|(% style="width:227px" %)G4Y\|(% style="width:527px" %)Growth rate, over 4 years\|(% style="width:976px" %)
	199	\|(% style="width:227px" %)GC5Y\|(% style="width:527px" %)Compound growth rate, over 5 years\|(% style="width:976px" %)
	200	\|(% style="width:227px" %)GC10Y\|(% style="width:527px" %)Compound growth rate, over 10 years\|(% style="width:976px" %)
	201	\|(% style="width:227px" %)GO1Y\|(% style="width:527px" %)Contribution to growth rate, over 1 year\|(% style="width:976px" %)
	202	\|(% style="width:227px" %)C1Y\|(% style="width:527px" %)Cumulated sum, over 1 year\|(% style="width:976px" %)
1.1	203
1.6	204	The use of codes like G3Y introduces redundancy in the code list. G3Y equals G36 for monthly data, G12 for quarterly data and G3 for annual data. Thus introducing such extensions should be well justified by solid use cases and DSD guidelines should explain which of the two possibilities (GxY or Gx) are preferred and why. Machine-to-machine queries, formulas, validation rules or coding templates may require mappings between those possibilities, taking into account both the frequency of a series and the transformation code.
1.1	205
2.13	206	= 4. Annex: coded examples =
1.1	207
	208	The table below shows coding example using all 3 options lined out above.
	209
3.1	210	\|(% style="width:571px" %)Statement\|(% style="width:486px" %)Normalised{{footnote}}For sake of readability the prefix TIMETRANS_ was not put in the table. The concepts are in fact called TIMETRANS_TYPE and TIMETRANS_PER.{{/footnote}}\|Type+Period\|Type+Period+Freq
	211	\|(% style="width:571px" %)Level series (non transformed data)\|(% style="width:486px" %)(((
1.1	212	FREQ=A or Q or M …
	213	TYPE=N
	214	PER=1
	215	)))\|(((
	216	FREQ=A or Q or M …
	217	TIMETRANS=N
	218	)))\|(((
	219	FREQ=A or Q or M …
	220	TIMETRANS=N
	221	)))
3.1	222	\|(% style="width:571px" %)Quarter on quarter growth rate\|(% style="width:486px" %)(((
1.1	223	FREQ=Q
	224	TYPE=G
	225	PER=1
	226	)))\|(((
	227	FREQ=Q
	228	TIMETRANS=G1
	229	)))\|(((
	230	FREQ=Q or M …
	231	TIMETRANS=G1Q
	232	)))
3.1	233	\|(% style="width:571px" %)Contribution to growth over 1 year (quarterly data)\|(% style="width:486px" %)(((
1.1	234	FREQ=Q
	235	TYPE=GC
	236	PER=4
	237	)))\|(((
	238	FREQ=Q
	239	TIMETRANS=GC4
	240	)))\|(((
	241	FREQ=Q
	242	TIMETRANS=GC1Y
	243	)))
3.1	244	\|(% style="width:571px" %)Contribution to growth over 1 year (annual data)\|(% style="width:486px" %)(((
1.1	245	FREQ=A
	246	TYPE=GC
	247	PER=1
	248	)))\|(((
	249	FREQ=A
	250	TIMETRANS=GC1
	251	)))\|(((
	252	FREQ=A
	253	TIMETRANS=GC1Y
	254	)))
3.1	255	\|(% style="width:571px" %)3 months moving average\|(% style="width:486px" %)(((
1.1	256	FREQ=M
3.1	257	~\|TYPE=A
1.1	258	PER=3
	259	)))\|(((
	260	FREQ=M
	261	TIMETRANS=A3
	262	)))\|(((
	263	FREQ=Q or M …
	264	TIMETRANS=A3M
	265	)))
3.1	266	\|(% style="width:571px" %)Annual index\|(% style="width:486px" %)(((
1.1	267	FREQ=A
	268	TYPE=I
	269	PER=1
	270	)))\|(((
	271	FREQ=A
	272	TIMETRANS=I1
	273	)))\|(((
	274	FREQ=A or Q or M …
	275	TIMETRANS=I1Y
	276	)))
	277
	278
	279	----
	280
2.3	281	{{putFootnotes/}}