Skip to Content
Version 1.3 by Helena K. on 2026/01/16 11:51
Hide last authors
Helena K. 1.1 1 {{box title="**Contents**"}}
2 {{toc/}}
3 {{/box}}
4
5 = 1. Problem statement =
6
Helena K. 1.3 7 In many cases, data that are exchanged in SDMX data messages do not relate to the calendar year. However, many statistical system implementations require that data are mapped to and stored as the real calendar.
Helena K. 1.1 8
9 This guideline provides recommendations for the following four use cases of such non-calendar year data:
10
11 1. Reporting year is equal to the calendar year
12 1. Reporting year starts on the first day of a month different to January
13 Example: school year starting on the 1^^st^^ of September, fiscal year starting on the 1^^st^^ of July
14 1. Reporting year starts on a given day in the year
15 Example: tax year starting on the 5^^th^^ of April
16 1. Reporting year **ends** on a given day in the year
17 Example: fiscal year ending on the 30^^th^^ of June, equivalent to fiscal year starting on the 1^^st^^ of July Y-1
18
19 It is also required to take into account reporting frequencies other than annual (e.g. quarterly, monthly). The use cases lined out above can be visualised graphically. Note that, in this example, all time spans would use the year 2015 (“2015” or “2015-A1”) as the reference year in reporting:
20
Helena K. 1.2 21 [[image:1768553211015-575.png]]
Helena K. 1.1 22
Helena K. 1.3 23 Assuming that the time dimension has the ID TIME_PERIOD, case 1 could use TIME_PERIOD=”2015”. All other cases would need to use the notation TIME_PERIOD=”2015-A1”. For the rest of the document we use the more general notation “2015-A1”.{{footnote}}The YYYY format is reserved for Gregorian years, i.e. from January 1 to December 31. For reporting years that are not Gregorian years, the format YYYY-A1 (e.g. 2016-A1) must be used for the time dimension.{{/footnote}}
Helena K. 1.1 24
25 For quarterly or monthly data, the first period of the reporting year would also need to be read as relative to the start of the reporting year. Here are examples for case 2 if the data is quarterly (2a) or monthly (2b):
26
Helena K. 1.2 27 [[image:1768553250696-532.png]]
Helena K. 1.1 28
29 Note that in case 2a, the third quarter of reporting year 2015 (TIME_PERIOD=2015-Q3) is covering data from March until June 2016. In case 2b, the seventh month of reporting year 2015 (TIME_PERIOD=2015-M07) is covering data for January 2016.
30
Helena K. 1.3 31 It is clear in the graphical view that time series under case 2b should not be directly compared with time series under case 4. If it is required to compare figures of the same time period, the 2015 observation of the type 2b series is compared with the 2016 figure of the type 4 series. Comparing a 2a series with others from the example may be more complicated, because exact alignment of the timelines might not be possible without additional data. It is possible to estimate aligned timelines by doing a time transformation using formulas; for instance, shifting the time series to comply with the calendar year. The methodological aspects of time transformation are not part of these guidelines. For coding the results of such transformations, please refer to the guidelines on coding time transformations in SDMX{{footnote}}https://sdmx.org/wp-content/uploads/SWG_TimeTransformation_V1.0.docx{{/footnote}}.
Helena K. 1.1 32
33 = 2. SDMX Concepts for Non-Calendar Year Series =
34
35 == //Start and end day coding// ==
36
37 In SDMX messages, the time period concept (concept ID TIME_PERIOD) specifies the **reporting period**. This reporting period, as outlined above, is in many cases not aligned with the calendar year.
38
Helena K. 1.3 39 To specify the calendar period that a reporting period is covering, the SDMX technical standard already defines an attribute “reporting year start” on series level with format xs:gMonthDay{{footnote}}https://sdmx.org/wp-content/uploads/SDMX_2-1-1-SECTION_6_TechnicalNotes-march-2013.pdf, row 658.{{/footnote}}. It gives a day and month when the reporting year starts. It is optional and if not provided the default is 1^^st^^ January. Using this attribute can cover cases 1, 2 and 3 from above. When added to a series, the attribute will specify on which day of the calendar year the reporting year starts.
Helena K. 1.1 40
41 This is not sufficient to cover case 4, because in this case the reporting year 2015 starts in 2014. In order to specify this case, a different attribute “reporting year end” should be used. It is also on series level with format xs:gMonthDay and gives a day and month when the reporting year ends. It is optional and if not provided the default is 31^^st^^ December.
42
43 The technical note (revision 1.0) suggests that the attribute ID for reporting year start day to be REPORTING_YEAR_START_DAY. However, to maintain EDI compliance, it is suggested to use the IDs REPYEARSTART and REPYEAREND when defining DSDs.
44
45 For any given series either start or end day should be used, but not both. This also implies that a reporting year will always have a duration of one year. In case reporting years do not last exactly a year, refer to the ISO time interval specification described in the next chapter. It is also strongly recommended not to use 29^^th^^ February as either start or end date of the reporting year, since that might lead to undefined situations.
46
47 == //ISO 8601 time interval// ==
48
Helena K. 1.3 49 As outlined above, in some cases reporting periods might not have the same duration as calendar periods. A crop year in agriculture may only last for a couple of months. To specify these periods on a more granular level, the SDMX technical standard suggests using the time intervals as defined by ISO 8601. ISO specifies four ways to express a time interval{{footnote}}http://www.iso.org/iso/home/standards/iso8601.htm and
50 https://sdmx.org/wp-content/uploads/SDMX_2-1-1-SECTION_6_TechnicalNotes-march-2013.pdf, row 749.{{/footnote}}:
Helena K. 1.1 51
52 1. Start and end, such as "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z"
53 1. Start and duration, such as "2007-03-01T13:00:00Z/P1Y2M10DT2H30M"
54 1. Duration and end, such as "P1Y2M10DT2H30M/2008-05-11T15:30:00Z"
55 1. Duration only, such as "P1Y2M10DT2H30M", with additional context information
56
57 An attribute time range (ID:TIME_RANGE) is suggested at the observation level to specify further in which particular time range a specific observation was collected. The series will still specify when the reporting period starts or ends for the whole series and each observation can have a specific time range within that reporting period. It is suggested for that case to use only option 1, the start and end date, and not the other options in SDMX context.
58
59 = 3. Example 1: Based on National Accounts data exchange =
60
61 Note that the SDMX fragments shown in the examples below are not syntactically correct. They serve the purpose of explaining the issue and show the coding in a readable way (pseudo code).
62
63 == //Current NA definition// ==
64
65 In the National Accounts Data Structure Definitions, fiscal year reporting was initially covered with the “reference period detail” attribute. This attribute is coded and only covers cases where a fiscal year starts on the first day of a given month. For other cases, a code for “other definition” was introduced.
66
67 The code list for “reference period detail” previously used in the National Accounts DSDs is as follows:
68
69 |**Code**|**Description**
70 |C|Calendar year
71 |F_O|Fiscal year (other definition)
72 |F02|Fiscal year starting in February
73 |F03|Fiscal year starting in March
74 |F04|Fiscal year starting in April
75 |F05|Fiscal year starting in May
76 |F06|Fiscal year starting in June
77 |F07|Fiscal year starting in July
78 |F08|Fiscal year starting in August
79 |F09|Fiscal year starting in September
80 |F10|Fiscal year starting in October
81 |F11|Fiscal year starting in November
82 |F12|Fiscal year starting in December
83
84
85
86 Following this code list, the cases lined out in the problem statement could be coded as such:
87
88 1. Reporting year is equal to the calendar year
89
90 <na_:Series **REF_PERIOD_DETAIL="C" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 91 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 92 </na_:Series>
93
94
95 REF_PERIOD_DETAIL="C" à reporting year 1995 starts 1^^st^^ January 1995
96
97 1. Reporting year starts on the first day of a month different to January
98 Example: fiscal year starting on the 1^^st^^ of July
99
100 <na_:Series **REF_PERIOD_DETAIL="F07" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 101 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 102 </na_:Series>
103
104
105 REF_PERIOD_DETAIL="F07" à reporting year 1995 starts 1^^st^^ July 1995 and Q1 projected on the calendar year goes from: 07-09/1995
106
107 1. Reporting year starts on a given day in the year
108 Example: tax year starting on the 5^^th^^ of April
109
110 <na_:Series **REF_PERIOD_DETAIL="F04" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 111 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 112 </na_:Series>
113
114
115 REF_PERIOD_DETAIL="F04" à reporting year 1995 would start 1^^st^^ April 1995. The case is not solved because we do not know anymore that it should be on the 5^^th^^. Another option would be to use the “F_O” code:
116
117 <na_:Series **REF_PERIOD_DETAIL="F_O" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 118 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 119 </na_:Series>
120
121
122 REF_PERIOD_DETAIL="F_O" à reporting year 1995 has another definition. The case is only partly solved because we do not know the exact date. Additional metadata are needed and the reporting period cannot directly be parsed by a system.
123
124 1. Reporting year **ends** on a given day in the year
125 Example: fiscal year ending on the 30^^th^^ of June, equivalent to fiscal year starting on the 1^^st^^ of July Y-1
126
127 <na_:Series **REF_PERIOD_DETAIL="??" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 128 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 129 </na_:Series>
130
131
132 REF_PERIOD_DETAIL="??" à the case is not covered. The code “F_O” could be used with the same problems as lined above for case 3.
133
134 == //Recommendations based on the use cases from National Accounts// ==
135
136 Using the attributes as suggested above, all cases outlined in the National Account examples can be fully covered:
137
138 1. Reporting year is equal to the calendar year
139
140 <na_:Series **REF_PERIOD_DETAIL="C" REPYEARSTART="~-~-01-01"// //**STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 141 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 142 </na_:Series>
143
144 1995-Q1 is the same as the calendar definition: from 1^^st^^ January 1995 until 31^^st^^ March 1995. In this case, the attribute REPYEARSTART may be omitted, since it expressed the default value of 1^^st^^ January.
145
146 Reporting year starts on the first day of a month different to January
147 Example: fiscal year starting on the 1^^st^^ of July
148
149 <na_:Series **REF_PERIOD_DETAIL="F07" REPYEARSTART="~-~-07-01" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 150 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 151 </na_:Series>
152
153 Reporting period 1995-Q1 lasts in that case from 1^^st^^ July 1995 until 30^^th^^ September 1995.
154
155 1. Reporting year starts on a given day in the year
156 Example: tax year starting on the 5^^th^^ of April
157
158 <na_:Series **REF_PERIOD_DETAIL="F04" REPYEARSTART="~-~-04-05" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 159 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 160 </na_:Series>
161
162 Reporting period 1995-Q1 lasts in that case from 5^^th^^ April 1995 until 4^^th^^ July 1995.
163
164 1. Reporting year **ends** on a given day in the year
165 Example: fiscal year ending on the 30^^th^^ of June, equivalent to fiscal year starting on the 1^^st^^ of July Y-1
166
167 <na_:Series **REF_PERIOD_DETAIL="??" REPYEAREND="~-~-06-30" **STO="B1G" REF_AREA="LU" FREQ="Q">
Helena K. 1.3 168 <na_:Obs OBS_VALUE="44" TIME_PERIOD="**1995**-Q1"/>
Helena K. 1.1 169 </na_:Series>
170
171 Reporting period 1995-Q1 lasts in that case from 1^^st^^ July 1994 until 30^^th^^ September 1994
172
173 1. Example 2: Based on Agriculture data exchange
174
175 == //Problem statement// ==
176
177 The problem statement explains some issues faced in agriculture statistics when reporting on non-calendar year observations. The examples are somewhat simplified to focus on the issue at hand.
178
179 In agriculture there are usually three numbers associated with crop production: the planted area, the harvested area and the production in quantity or value. From this, yield may be derived in order to perform economic calculations and use it for other purposes.
180
181 Let us take the example of wheat. In most cases an area is planted in year n and it is harvested in year n+1. As can be seen in the examples below about wheat growing periods in several regions in the world, the reference period usually refers to the year of the harvest (year n+1). From these observations, different indicators may be derived and the relation be made that the planted area of year n is to the production of year n+1.
182
183 To be able to do this without mistakes when looking at the three datasets mentioned (harvested area, planted area, production), the start date, the end date and the year reported by the country are required for each dataset for comparability reasons. The Start date and end date of the observation will allow the data to be linked to perform calculations and analysis. The reported year is important to determine how the country reports and publishes its data and to compare and explain differences between the published data of each country and, for instance, data published by an international organisation.
184
185 Also, the growing periods for wheat may change according to the weather. Sometimes the planting and harvesting may happen during the same calendar year and sometimes not. The growing period might be a little shorter or longer. It would be good to know for each year according to the length of the growing period how the country codes the data: when the season goes beyond 31 Dec, they might use year n or year n+1.
186
187 In many countries, data are published with the year written as periods (e.g. 2009-10 for annual data about the 2009-2010 period). This notation is also commonly used in education statistics, but is not part of the standard observational time-period format defined in SDMX.
188
189 == //Proof of concept DSD// ==
190
191 Note that the DSD proposal below does not constitute a real usable DSD for agriculture statistics. It is heavily simplified to only include concepts that are relevant for the purpose of these guidelines. Also the SDMX fragments shown in the example are not syntactically correct. They just serve the purpose of explaining the issue and show the coding in a readable way (pseudo code).
192
193 |**Concept ID**|**Description**|**Role**|**Code List / Format**
194 |**FREQ**|Frequency|Dimension|A (annual), S (Half-yearly)
195 |**REF_AREA**|Reference area|Dimension|IN (India), CN (China)
196 |**CROP**|Crop|Dimension|W (Wheat)
197 | |Crop
198 (alternative coding)|Dimension|W (Total Wheat), WW (winter wheat), SW (summer wheat)
199 |**ACTIVITY**|Activity|Dimension|P (planting), H (harvesting)
200 |**REPYEARSTART**|Reference year start|Attribute
201 (optional, series)|xs:gMonthDay
202 |**REPYEAREND**|Reference year end|Attribute
203 (optional, series)|xs:gMonthDay
204
205 == //Usage of guidelines// ==
206
207 === India ===
208
209 Planting: wheat takes place in October to December (e.g. 2010)
210
211 Harvest: during the months of February and May (e.g. 2011).
212
213 Assumption: India attributes both activities to “crop year” 2011, which goes from Oct 2010-May 2011
214
215 **Example data:**
216
217 Oct-Dec 2010: 100 units planted
218
219 Feb-May 2011: 100 units harvested
220
221 Oct-Dec 2011: 200 units planted
222
223 Feb-May 2012: 200 units harvested
224
225 **Coding:**
226
227 <Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=P **REPYEAREND=~-~-05-31**>
228 Obs TIME_PERIOD=2011-A1 VALUE=100
229 Obs TIME_PERIOD=2012-A1 VALUE=200
230
231
232 <Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=H **REPYEAREND=~-~-05-31**>
233 Obs TIME_PERIOD=2011-A1 VALUE=100
234 Obs TIME_PERIOD=2012-A1 VALUE=200
235
236
237 The end of the reporting year is the end of May (attribute REPYEAREND). By definition, the year 2011 would thus go from 1^^st^^ June 2010 – 31^^st^^ May 2011. The real crop year is shorter, but since it is the same for every year and always within that time range, this information is sufficient to be available as reference metadata.
238
239 === China ===
240
241 Winter wheat is planted from mid-September through October and harvested from mid-May through June. Summer wheat is planted from mid-March through April and harvested from mid-July to mid-August.
242
243 Assumption: China attributes both activities to a “crop year”, which goes from September of the previous until May of the current year.
244
245 **Sample Data for China**
246
247 |**Crop year**|**Season**|**Calendar period**|**Units**
248 |**2011**|Winter|Sep-Oct 2010|100 planted
249 | | |May-Jun 2011|100 harvested
250 | |Summer|Mar-Apr 2011|200 planted
251 | | |Jul-Aug 2011|200 harvested
252 |**2012**|Winter|Sep-Oct 2011|120 planted
253 | | |May-Jun 2011|120 harvested
254 | |Summer|Mar-Apr 2012|220 planted
255 | | |Jul-Aug 2012|220 harvested
256
257 == //Recommendations based on the Use Cases from Agriculture// ==
258
259 For coding the SDMX message, the frequency of the data is half-yearly, since we have two seasons per year. The reporting year end (31^^st^^ August) corresponds to the whole reporting year, ranging from 1^^st^^ September Y-1 until 31^^st^^ August Y. The exact duration of the season would, in this example, only be available in metadata.
260
261 <Series FREQ=S REF_AREA=CN CROP=W ACTIVITY=P REPYEAREND=~-~-08-31>
262 Obs TIME_PERIOD=2011-S1 VALUE=100
263 Obs TIME_PERIOD=2011-S2 VALUE=200
264 Obs TIME_PERIOD=2012-S1 VALUE=120
265 Obs TIME_PERIOD=2012-S2 VALUE=220
266
267
268 <Series FREQ=S REF_AREA=CN CROP=W ACTIVITY=H REPYEAREND=~-~-08-31>
269 Obs TIME_PERIOD=2011-S1 VALUE=100
270 Obs TIME_PERIOD=2011-S2 VALUE=200
271 Obs TIME_PERIOD=2012-S1 VALUE=120
272 Obs TIME_PERIOD=2012-S2 VALUE=220
273
274
275 To calculate the total annual planting of crop, the half-yearly series are added up to annual series (2011=2011-S1+2011-S2 à P: 100+200=300; H: 100+200=300).
276
277 <Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=P REPYEAREND=~-~-08-31>
278 Obs TIME_PERIOD=2011-A1 VALUE=300
279 Obs TIME_PERIOD=2012-A1 VALUE=340
280 <Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=H REPYEAREND=~-~-08-31>
281 Obs TIME_PERIOD=2011-A1 VALUE=300
282 Obs TIME_PERIOD=2012-A1 VALUE=340
283
284
285 If the alternative coding of Winter Wheat (WW) and Summer Wheat (SW) is used, the frequency becomes annual and values can also be added up separately:
286
287 <Series FREQ=A REF_AREA=CN CROP=WW ACTIVITY=P REPYEAREND=~-~-08-31>
288 Obs TIME_PERIOD=2011-A1 VALUE=100
289 Obs TIME_PERIOD=2012-A1 VALUE=120
290
291
292 <Series FREQ=A REF_AREA=CN CROP=WW ACTIVITY=H REPYEAREND=~-~-08-31>
293 Obs TIME_PERIOD=2011-A1 VALUE=100
294 Obs TIME_PERIOD=2012-A1 VALUE=120
295
296
297 <Series FREQ=A REF_AREA=CN CROP=SW ACTIVITY=P REPYEAREND=~-~-08-31>
298 Obs TIME_PERIOD=2011-A1 VALUE=200
299 Obs TIME_PERIOD=2012-A1 VALUE=220
300
301
302 <Series FREQ=A REF_AREA=CN CROP=SW ACTIVITY=H REPYEAREND=~-~-08-31>
303 Obs TIME_PERIOD=2011-A1 VALUE=200
304 Obs TIME_PERIOD=2012-A1 VALUE=220
305
306
307 For the total annual planting of crop, the winter and summer wheat can be added (W=WW+SW) and lead to the same result as above:
308
309 <Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=P REPYEAREND=~-~-08-31>
310 Obs TIME_PERIOD=2011-A1 VALUE=300
311 Obs TIME_PERIOD=2012-A1 VALUE=340
312 <Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=H REPYEAREND=~-~-08-31>
313 Obs TIME_PERIOD=2011-A1 VALUE=300
314 Obs TIME_PERIOD=2012-A1 VALUE=340
315
316 === Date range coding example ===
317
318 As it can be seen in the examples above, the exact period when the actual planting / harvesting took place can usually be expressed as reference metadata. However, when adding up figures from different reporting year definitions (e.g. adding India and China for 2011), one needs to pay attention to the different definitions. There is no implicit SDMX solution, because SDMX only expresses the data but does not contain the methodology of how data are added for different reporting periods. If more detailed information is needed; for instance, an algorithm to attribute crop production to exact comparable periods, the exact time ranges need to be known as input to the estimation formulas.
319
320 To take the examples of India from above, slightly modified:
321
322 1st Oct- 27th Dec 2010: 100 units planted
323
324 5th Feb- 17th May 2011: 100 units harvested
325
326 15th Oct- 2nd Jan 2012: 200 units planted
327
328 7th Feb-5th May 2012: 200 units harvested
329
330 It can be coded as follows, to give the data user the maximum information for further processing:
331
332 <Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=P REPYEAREND=~-~-05-31>
333 Obs TIME_PERIOD=2011-A1 VALUE=100 TIME_RANGE=2010-10-01/2010-12-27
334 Obs TIME_PERIOD=2012-A1 VALUE=200 TIME_RANGE=2011-10-15/2012-01-02
335
336
337 <Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=H REPYEAREND=~-~-05-31>
338 Obs TIME_PERIOD=2011-A1 VALUE=100 TIME_RANGE=2011-02-05/2011-05-17
339 Obs TIME_PERIOD=2012-A1 VALUE=200 TIME_RANGE=2012-02-07/2012-05-05
340
341
342 Care needs to be taken that the time range for each observation is within the period covered by the reporting period projected on the calendar. For instance, a harvesting range lasting from the beginning of March until the end of June would be invalid, because the reporting period ends at the end of May. In case harvesting would last until June, the reporting year end would need to be changed accordingly.
343
344 ----
345
Helena K. 1.3 346 {{putFootnotes/}}
© Semantic R&D Group, 2026