Wiki source code of Guidelines on Non-Calendar Year Reporting of Data

Version 1.2 by Helena K. on 2026/01/16 11:47

version	line-number	content
1.1	1	{{box title="Contents"}}
	2	{{toc/}}
	3	{{/box}}
	4
	5	= 1. Problem statement =
	6
	7	In many cases, data that are exchanged in SDMX data messages do not relate to the calendar year. However, many statistical system implementations require that data are mapped to and stored as the real calendar.
	8
	9	This guideline provides recommendations for the following four use cases of such non-calendar year data:
	10
	11	1. Reporting year is equal to the calendar year
	12	1. Reporting year starts on the first day of a month different to January
	13	Example: school year starting on the 1^^st^^ of September, fiscal year starting on the 1^^st^^ of July
	14	1. Reporting year starts on a given day in the year
	15	Example: tax year starting on the 5^^th^^ of April
	16	1. Reporting year ends on a given day in the year
	17	Example: fiscal year ending on the 30^^th^^ of June, equivalent to fiscal year starting on the 1^^st^^ of July Y-1
	18
	19	It is also required to take into account reporting frequencies other than annual (e.g. quarterly, monthly). The use cases lined out above can be visualised graphically. Note that, in this example, all time spans would use the year 2015 (“2015” or “2015-A1”) as the reference year in reporting:
	20
1.2	21	[[image:1768553211015-575.png]]
1.1	22
1.2	23	Assuming that the time dimension has the ID TIME_PERIOD, case 1 could use TIME_PERIOD=”2015”. All other cases would need to use the notation TIME_PERIOD=”2015-A1”. For the rest of the document we use the more general notation “2015-A1”. [[~[1~]>>path:#_ftn1\|\|style="background-color: rgb(255, 255, 255);"]]
1.1	24
	25	For quarterly or monthly data, the first period of the reporting year would also need to be read as relative to the start of the reporting year. Here are examples for case 2 if the data is quarterly (2a) or monthly (2b):
	26
1.2	27	[[image:1768553250696-532.png]]
1.1	28
	29	Note that in case 2a, the third quarter of reporting year 2015 (TIME_PERIOD=2015-Q3) is covering data from March until June 2016. In case 2b, the seventh month of reporting year 2015 (TIME_PERIOD=2015-M07) is covering data for January 2016.
	30
	31	It is clear in the graphical view that time series under case 2b should not be directly compared with time series under case 4. If it is required to compare figures of the same time period, the 2015 observation of the type 2b series is compared with the 2016 figure of the type 4 series. Comparing a 2a series with others from the example may be more complicated, because exact alignment of the timelines might not be possible without additional data. It is possible to estimate aligned timelines by doing a time transformation using formulas; for instance, shifting the time series to comply with the calendar year. The methodological aspects of time transformation are not part of these guidelines. For coding the results of such transformations, please refer to the guidelines on coding time transformations in SDMX[[~[2~]>>path:#_ftn2]].
	32
	33	= 2. SDMX Concepts for Non-Calendar Year Series =
	34
	35	== //Start and end day coding// ==
	36
	37	In SDMX messages, the time period concept (concept ID TIME_PERIOD) specifies the reporting period. This reporting period, as outlined above, is in many cases not aligned with the calendar year.
	38
	39	To specify the calendar period that a reporting period is covering, the SDMX technical standard already defines an attribute “reporting year start” on series level with format xs:gMonthDay[[~[3~]>>path:#_ftn3]]. It gives a day and month when the reporting year starts. It is optional and if not provided the default is 1^^st^^ January. Using this attribute can cover cases 1, 2 and 3 from above. When added to a series, the attribute will specify on which day of the calendar year the reporting year starts.
	40
	41	This is not sufficient to cover case 4, because in this case the reporting year 2015 starts in 2014. In order to specify this case, a different attribute “reporting year end” should be used. It is also on series level with format xs:gMonthDay and gives a day and month when the reporting year ends. It is optional and if not provided the default is 31^^st^^ December.
	42
	43	The technical note (revision 1.0) suggests that the attribute ID for reporting year start day to be REPORTING_YEAR_START_DAY. However, to maintain EDI compliance, it is suggested to use the IDs REPYEARSTART and REPYEAREND when defining DSDs.
	44
	45	For any given series either start or end day should be used, but not both. This also implies that a reporting year will always have a duration of one year. In case reporting years do not last exactly a year, refer to the ISO time interval specification described in the next chapter. It is also strongly recommended not to use 29^^th^^ February as either start or end date of the reporting year, since that might lead to undefined situations.
	46
	47	== //ISO 8601 time interval// ==
	48
	49	As outlined above, in some cases reporting periods might not have the same duration as calendar periods. A crop year in agriculture may only last for a couple of months. To specify these periods on a more granular level, the SDMX technical standard suggests using the time intervals as defined by ISO 8601. ISO specifies four ways to express a time interval[[~[4~]>>path:#_ftn4]]:
	50
	51	1. Start and end, such as "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z"
	52	1. Start and duration, such as "2007-03-01T13:00:00Z/P1Y2M10DT2H30M"
	53	1. Duration and end, such as "P1Y2M10DT2H30M/2008-05-11T15:30:00Z"
	54	1. Duration only, such as "P1Y2M10DT2H30M", with additional context information
	55
	56
	57
	58	An attribute time range (ID:TIME_RANGE) is suggested at the observation level to specify further in which particular time range a specific observation was collected. The series will still specify when the reporting period starts or ends for the whole series and each observation can have a specific time range within that reporting period. It is suggested for that case to use only option 1, the start and end date, and not the other options in SDMX context.
	59
	60	= 3. Example 1: Based on National Accounts data exchange =
	61
	62	Note that the SDMX fragments shown in the examples below are not syntactically correct. They serve the purpose of explaining the issue and show the coding in a readable way (pseudo code).
	63
	64	== //Current NA definition// ==
	65
	66	In the National Accounts Data Structure Definitions, fiscal year reporting was initially covered with the “reference period detail” attribute. This attribute is coded and only covers cases where a fiscal year starts on the first day of a given month. For other cases, a code for “other definition” was introduced.
	67
	68	The code list for “reference period detail” previously used in the National Accounts DSDs is as follows:
	69
	70	\|Code\|Description
	71	\|C\|Calendar year
	72	\|F_O\|Fiscal year (other definition)
	73	\|F02\|Fiscal year starting in February
	74	\|F03\|Fiscal year starting in March
	75	\|F04\|Fiscal year starting in April
	76	\|F05\|Fiscal year starting in May
	77	\|F06\|Fiscal year starting in June
	78	\|F07\|Fiscal year starting in July
	79	\|F08\|Fiscal year starting in August
	80	\|F09\|Fiscal year starting in September
	81	\|F10\|Fiscal year starting in October
	82	\|F11\|Fiscal year starting in November
	83	\|F12\|Fiscal year starting in December
	84
	85
	86
	87	Following this code list, the cases lined out in the problem statement could be coded as such:
	88
	89	1. Reporting year is equal to the calendar year
	90
	91	<na_:Series REF_PERIOD_DETAIL="C" STO="B1G" REF_AREA="LU" FREQ="Q">
	92	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	93	</na_:Series>
	94
	95
	96	REF_PERIOD_DETAIL="C" à reporting year 1995 starts 1^^st^^ January 1995
	97
	98	1. Reporting year starts on the first day of a month different to January
	99	Example: fiscal year starting on the 1^^st^^ of July
	100
	101	<na_:Series REF_PERIOD_DETAIL="F07" STO="B1G" REF_AREA="LU" FREQ="Q">
	102	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	103	</na_:Series>
	104
	105
	106	REF_PERIOD_DETAIL="F07" à reporting year 1995 starts 1^^st^^ July 1995 and Q1 projected on the calendar year goes from: 07-09/1995
	107
	108	1. Reporting year starts on a given day in the year
	109	Example: tax year starting on the 5^^th^^ of April
	110
	111	<na_:Series REF_PERIOD_DETAIL="F04" STO="B1G" REF_AREA="LU" FREQ="Q">
	112	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	113	</na_:Series>
	114
	115
	116	REF_PERIOD_DETAIL="F04" à reporting year 1995 would start 1^^st^^ April 1995. The case is not solved because we do not know anymore that it should be on the 5^^th^^. Another option would be to use the “F_O” code:
	117
	118	<na_:Series REF_PERIOD_DETAIL="F_O" STO="B1G" REF_AREA="LU" FREQ="Q">
	119	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	120	</na_:Series>
	121
	122
	123	REF_PERIOD_DETAIL="F_O" à reporting year 1995 has another definition. The case is only partly solved because we do not know the exact date. Additional metadata are needed and the reporting period cannot directly be parsed by a system.
	124
	125	1. Reporting year ends on a given day in the year
	126	Example: fiscal year ending on the 30^^th^^ of June, equivalent to fiscal year starting on the 1^^st^^ of July Y-1
	127
	128	<na_:Series REF_PERIOD_DETAIL="??" STO="B1G" REF_AREA="LU" FREQ="Q">
	129	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	130	</na_:Series>
	131
	132
	133	REF_PERIOD_DETAIL="??" à the case is not covered. The code “F_O” could be used with the same problems as lined above for case 3.
	134
	135	== //Recommendations based on the use cases from National Accounts// ==
	136
	137	Using the attributes as suggested above, all cases outlined in the National Account examples can be fully covered:
	138
	139	1. Reporting year is equal to the calendar year
	140
	141	<na_:Series REF_PERIOD_DETAIL="C" REPYEARSTART="~-~-01-01"// //STO="B1G" REF_AREA="LU" FREQ="Q">
	142	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	143	</na_:Series>
	144
	145
	146	1995-Q1 is the same as the calendar definition: from 1^^st^^ January 1995 until 31^^st^^ March 1995. In this case, the attribute REPYEARSTART may be omitted, since it expressed the default value of 1^^st^^ January.
	147
	148	Reporting year starts on the first day of a month different to January
	149	Example: fiscal year starting on the 1^^st^^ of July
	150
	151	<na_:Series REF_PERIOD_DETAIL="F07" REPYEARSTART="~-~-07-01" STO="B1G" REF_AREA="LU" FREQ="Q">
	152	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	153	</na_:Series>
	154
	155	Reporting period 1995-Q1 lasts in that case from 1^^st^^ July 1995 until 30^^th^^ September 1995.
	156
	157	1. Reporting year starts on a given day in the year
	158	Example: tax year starting on the 5^^th^^ of April
	159
	160	<na_:Series REF_PERIOD_DETAIL="F04" REPYEARSTART="~-~-04-05" STO="B1G" REF_AREA="LU" FREQ="Q">
	161	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	162	</na_:Series>
	163
	164	Reporting period 1995-Q1 lasts in that case from 5^^th^^ April 1995 until 4^^th^^ July 1995.
	165
	166	1. Reporting year ends on a given day in the year
	167	Example: fiscal year ending on the 30^^th^^ of June, equivalent to fiscal year starting on the 1^^st^^ of July Y-1
	168
	169	<na_:Series REF_PERIOD_DETAIL="??" REPYEAREND="~-~-06-30" STO="B1G" REF_AREA="LU" FREQ="Q">
	170	<na_:Obs OBS_VALUE="44" TIME_PERIOD="1995-Q1"/>
	171	</na_:Series>
	172
	173	Reporting period 1995-Q1 lasts in that case from 1^^st^^ July 1994 until 30^^th^^ September 1994
	174
	175	1. Example 2: Based on Agriculture data exchange
	176
	177	== //Problem statement// ==
	178
	179	The problem statement explains some issues faced in agriculture statistics when reporting on non-calendar year observations. The examples are somewhat simplified to focus on the issue at hand.
	180
	181	In agriculture there are usually three numbers associated with crop production: the planted area, the harvested area and the production in quantity or value. From this, yield may be derived in order to perform economic calculations and use it for other purposes.
	182
	183	Let us take the example of wheat. In most cases an area is planted in year n and it is harvested in year n+1. As can be seen in the examples below about wheat growing periods in several regions in the world, the reference period usually refers to the year of the harvest (year n+1). From these observations, different indicators may be derived and the relation be made that the planted area of year n is to the production of year n+1.
	184
	185	To be able to do this without mistakes when looking at the three datasets mentioned (harvested area, planted area, production), the start date, the end date and the year reported by the country are required for each dataset for comparability reasons. The Start date and end date of the observation will allow the data to be linked to perform calculations and analysis. The reported year is important to determine how the country reports and publishes its data and to compare and explain differences between the published data of each country and, for instance, data published by an international organisation.
	186
	187	Also, the growing periods for wheat may change according to the weather. Sometimes the planting and harvesting may happen during the same calendar year and sometimes not. The growing period might be a little shorter or longer. It would be good to know for each year according to the length of the growing period how the country codes the data: when the season goes beyond 31 Dec, they might use year n or year n+1.
	188
	189	In many countries, data are published with the year written as periods (e.g. 2009-10 for annual data about the 2009-2010 period). This notation is also commonly used in education statistics, but is not part of the standard observational time-period format defined in SDMX.
	190
	191	== //Proof of concept DSD// ==
	192
	193	Note that the DSD proposal below does not constitute a real usable DSD for agriculture statistics. It is heavily simplified to only include concepts that are relevant for the purpose of these guidelines. Also the SDMX fragments shown in the example are not syntactically correct. They just serve the purpose of explaining the issue and show the coding in a readable way (pseudo code).
	194
	195
	196	\|Concept ID\|Description\|Role\|Code List / Format
	197	\|FREQ\|Frequency\|Dimension\|A (annual), S (Half-yearly)
	198	\|REF_AREA\|Reference area\|Dimension\|IN (India), CN (China)
	199	\|CROP\|Crop\|Dimension\|W (Wheat)
	200	\| \|Crop
	201	(alternative coding)\|Dimension\|W (Total Wheat), WW (winter wheat), SW (summer wheat)
	202	\|ACTIVITY\|Activity\|Dimension\|P (planting), H (harvesting)
	203	\|REPYEARSTART\|Reference year start\|Attribute
	204	(optional, series)\|xs:gMonthDay
	205	\|REPYEAREND\|Reference year end\|Attribute
	206	(optional, series)\|xs:gMonthDay
	207
	208	== //Usage of guidelines// ==
	209
	210	=== India ===
	211
	212	Planting: wheat takes place in October to December (e.g. 2010)
	213
	214	Harvest: during the months of February and May (e.g. 2011).
	215
	216	Assumption: India attributes both activities to “crop year” 2011, which goes from Oct 2010-May 2011
	217
	218	Example data:
	219
	220	Oct-Dec 2010: 100 units planted
	221
	222	Feb-May 2011: 100 units harvested
	223
	224	Oct-Dec 2011: 200 units planted
	225
	226	Feb-May 2012: 200 units harvested
	227
	228	Coding:
	229
	230	<Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=P REPYEAREND=~-~-05-31>
	231	Obs TIME_PERIOD=2011-A1 VALUE=100
	232	Obs TIME_PERIOD=2012-A1 VALUE=200
	233
	234
	235	<Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=H REPYEAREND=~-~-05-31>
	236	Obs TIME_PERIOD=2011-A1 VALUE=100
	237	Obs TIME_PERIOD=2012-A1 VALUE=200
	238
	239
	240	The end of the reporting year is the end of May (attribute REPYEAREND). By definition, the year 2011 would thus go from 1^^st^^ June 2010 – 31^^st^^ May 2011. The real crop year is shorter, but since it is the same for every year and always within that time range, this information is sufficient to be available as reference metadata.
	241
	242	=== China ===
	243
	244	Winter wheat is planted from mid-September through October and harvested from mid-May through June. Summer wheat is planted from mid-March through April and harvested from mid-July to mid-August.
	245
	246	Assumption: China attributes both activities to a “crop year”, which goes from September of the previous until May of the current year.
	247
	248	Sample Data for China
	249
	250	\|Crop year\|Season\|Calendar period\|Units
	251	\|2011\|Winter\|Sep-Oct 2010\|100 planted
	252	\| \| \|May-Jun 2011\|100 harvested
	253	\| \|Summer\|Mar-Apr 2011\|200 planted
	254	\| \| \|Jul-Aug 2011\|200 harvested
	255	\|2012\|Winter\|Sep-Oct 2011\|120 planted
	256	\| \| \|May-Jun 2011\|120 harvested
	257	\| \|Summer\|Mar-Apr 2012\|220 planted
	258	\| \| \|Jul-Aug 2012\|220 harvested
	259
	260	== //Recommendations based on the Use Cases from Agriculture// ==
	261
	262	For coding the SDMX message, the frequency of the data is half-yearly, since we have two seasons per year. The reporting year end (31^^st^^ August) corresponds to the whole reporting year, ranging from 1^^st^^ September Y-1 until 31^^st^^ August Y. The exact duration of the season would, in this example, only be available in metadata.
	263
	264	<Series FREQ=S REF_AREA=CN CROP=W ACTIVITY=P REPYEAREND=~-~-08-31>
	265	Obs TIME_PERIOD=2011-S1 VALUE=100
	266	Obs TIME_PERIOD=2011-S2 VALUE=200
	267	Obs TIME_PERIOD=2012-S1 VALUE=120
	268	Obs TIME_PERIOD=2012-S2 VALUE=220
	269
	270
	271	<Series FREQ=S REF_AREA=CN CROP=W ACTIVITY=H REPYEAREND=~-~-08-31>
	272	Obs TIME_PERIOD=2011-S1 VALUE=100
	273	Obs TIME_PERIOD=2011-S2 VALUE=200
	274	Obs TIME_PERIOD=2012-S1 VALUE=120
	275	Obs TIME_PERIOD=2012-S2 VALUE=220
	276
	277
	278	To calculate the total annual planting of crop, the half-yearly series are added up to annual series (2011=2011-S1+2011-S2 à P: 100+200=300; H: 100+200=300).
	279
	280	<Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=P REPYEAREND=~-~-08-31>
	281	Obs TIME_PERIOD=2011-A1 VALUE=300
	282	Obs TIME_PERIOD=2012-A1 VALUE=340
	283	<Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=H REPYEAREND=~-~-08-31>
	284	Obs TIME_PERIOD=2011-A1 VALUE=300
	285	Obs TIME_PERIOD=2012-A1 VALUE=340
	286
	287
	288	If the alternative coding of Winter Wheat (WW) and Summer Wheat (SW) is used, the frequency becomes annual and values can also be added up separately:
	289
	290	<Series FREQ=A REF_AREA=CN CROP=WW ACTIVITY=P REPYEAREND=~-~-08-31>
	291	Obs TIME_PERIOD=2011-A1 VALUE=100
	292	Obs TIME_PERIOD=2012-A1 VALUE=120
	293
	294
	295	<Series FREQ=A REF_AREA=CN CROP=WW ACTIVITY=H REPYEAREND=~-~-08-31>
	296	Obs TIME_PERIOD=2011-A1 VALUE=100
	297	Obs TIME_PERIOD=2012-A1 VALUE=120
	298
	299
	300	<Series FREQ=A REF_AREA=CN CROP=SW ACTIVITY=P REPYEAREND=~-~-08-31>
	301	Obs TIME_PERIOD=2011-A1 VALUE=200
	302	Obs TIME_PERIOD=2012-A1 VALUE=220
	303
	304
	305	<Series FREQ=A REF_AREA=CN CROP=SW ACTIVITY=H REPYEAREND=~-~-08-31>
	306	Obs TIME_PERIOD=2011-A1 VALUE=200
	307	Obs TIME_PERIOD=2012-A1 VALUE=220
	308
	309
	310	For the total annual planting of crop, the winter and summer wheat can be added (W=WW+SW) and lead to the same result as above:
	311
	312	<Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=P REPYEAREND=~-~-08-31>
	313	Obs TIME_PERIOD=2011-A1 VALUE=300
	314	Obs TIME_PERIOD=2012-A1 VALUE=340
	315	<Series FREQ=A REF_AREA=CN CROP=W ACTIVITY=H REPYEAREND=~-~-08-31>
	316	Obs TIME_PERIOD=2011-A1 VALUE=300
	317	Obs TIME_PERIOD=2012-A1 VALUE=340
	318
	319	=== Date range coding example ===
	320
	321	As it can be seen in the examples above, the exact period when the actual planting / harvesting took place can usually be expressed as reference metadata. However, when adding up figures from different reporting year definitions (e.g. adding India and China for 2011), one needs to pay attention to the different definitions. There is no implicit SDMX solution, because SDMX only expresses the data but does not contain the methodology of how data are added for different reporting periods. If more detailed information is needed; for instance, an algorithm to attribute crop production to exact comparable periods, the exact time ranges need to be known as input to the estimation formulas.
	322
	323	To take the examples of India from above, slightly modified:
	324
	325	1st Oct- 27th Dec 2010: 100 units planted
	326
	327	5th Feb- 17th May 2011: 100 units harvested
	328
	329	15th Oct- 2nd Jan 2012: 200 units planted
	330
	331	7th Feb-5th May 2012: 200 units harvested
	332
	333	It can be coded as follows, to give the data user the maximum information for further processing:
	334
	335	<Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=P REPYEAREND=~-~-05-31>
	336	Obs TIME_PERIOD=2011-A1 VALUE=100 TIME_RANGE=2010-10-01/2010-12-27
	337	Obs TIME_PERIOD=2012-A1 VALUE=200 TIME_RANGE=2011-10-15/2012-01-02
	338
	339
	340	<Series FREQ=A REF_AREA=IN CROP=W ACTIVITY=H REPYEAREND=~-~-05-31>
	341	Obs TIME_PERIOD=2011-A1 VALUE=100 TIME_RANGE=2011-02-05/2011-05-17
	342	Obs TIME_PERIOD=2012-A1 VALUE=200 TIME_RANGE=2012-02-07/2012-05-05
	343
	344
	345	Care needs to be taken that the time range for each observation is within the period covered by the reporting period projected on the calendar. For instance, a harvesting range lasting from the beginning of March until the end of June would be invalid, because the reporting period ends at the end of May. In case harvesting would last until June, the reporting year end would need to be changed accordingly.
	346
	347	----
	348
	349	[[~[1~]>>path:#_ftnref1]] The //YYYY// format is reserved for Gregorian years, i.e. from January 1 to December 31. For reporting years that are not Gregorian years, the format YYYY-A1 (e.g. 2016-A1) must be used for the time dimension
	350
	351	[[~[2~]>>path:#_ftnref2]] [[https:~~/~~/sdmx.org/wp-content/uploads/SWG_TimeTransformation_V1.0.docx>>url:https://sdmx.org/wp-content/uploads/SWG_TimeTransformation_V1.0.docx]]
	352
	353	[[~[3~]>>path:#_ftnref3]] [[https:~~/~~/sdmx.org/wp-content/uploads/SDMX_2-1-1-SECTION_6_TechnicalNotes-march-2013.pdf>>url:https://sdmx.org/wp-content/uploads/SDMX_2-1-1-SECTION_6_TechnicalNotes-march-2013.pdf]], row 658
	354
	355	[[~[4~]>>path:#_ftnref4]] [[http:~~/~~/www.iso.org/iso/home/standards/iso8601.htm>>url:http://www.iso.org/iso/home/standards/iso8601.htm]] and
	356	[[https:~~/~~/sdmx.org/wp-content/uploads/SDMX_2-1-1-SECTION_6_TechnicalNotes-march-2013.pdf>>url:https://sdmx.org/wp-content/uploads/SDMX_2-1-1-SECTION_6_TechnicalNotes-march-2013.pdf]], row 749.