This post is about my efforts over the last four years to get three important documents relating to the Literacy and Numeracy Test for Initial Teacher Education students (LANTITE) released publicly.
Documents 1, 2 and 3
In September 2020, I learned via a Sydney Morning Herald article of a May 2020 report on the implementation of LANTITE received by the Department of Education, Skills and Employment (DESE). To get a copy of the report, I submitted a request to DESE under the Freedom of Information (FOI) Act and DESE released the report shortly afterwards. The Appendix to that report, ‘Our methodology’, suggests that Dandolo Partners, the private firm who produced the report, received “data on test participation and performance” from DESE and the Australian Council for Educational Research (ACER), who design and administer LANTITE.
This was important because it pointed to key documents which were presumed to exist but were not publicly available: technical and administration reports (more about the purpose and content of these below).
So, on November 24 2020, I submitted another FOI request to DESE, this time for the “data on LANTITE test participation and performance sent to dandolopartners in April 2019.” In December 2020, DESE advised me that it would cost me $450 to process my request (I paid this) and that it needed to
consult with certain third parties because the requested documents contain information that third parties might wish to contend should not be disclosed.
In January 2021, a Senior Government Lawyer wrote to me and explained that “the department has in its possession three documents consisting of 177 pages” that fell within the scope of my request. These were listed as Documents 1, 2 and 3 in a ‘Schedule of Documents’, alongside with the DESE’s decision regarding access to each document.
So, DESE had decided not to release any part of the Documents 1, 2 and 3. But at least I had further confirmation of the existence of technical reports and administration reports.
What are technical and administration reports?
Technical reports are routinely produced and released by testing organisations to such publication is seen within the educational measurement profession as meeting an ethical obligation to
provide evidence that the technical quality, including reliability and validity, of the test meets its intended purpose (Cizek and Rosenberg, 2011, p. 236, quoting the Code of Fair Testing Practices in Education, JCTP, 1998).
Take NAPLAN, for example. Technical reports are available here for each year since 2013. The 2013 Technical Report includes this explanation of its aims:
Best practice for educational assessment programs dictates public reporting of results and the statistical and psychometric analyses that led to these results. Commonly, these analyses are summarised in an annual Technical Report. A well-documented technical report provides the transparency that measurement professionals, educators, and the general public expect and builds confidence that the results of such assessments are reliable, valid, and fair. (p. 1)
Administration reports discuss the circumstances under which test-takers took the test in question. It includes accounts of incidents which may impact on the interpretation of results such as hardware or software failure, evacuations or other types of emergencies.
DESE’s reasons for refusing access to documents 1, 2 and 3
As the Schedule of Documents above shows, DESE gave three main reasons for refusing access to LANTITE reports (documents 1-3). Each of these relates to a particular subsection of the FOI Act.
First, DESE argued that “certain pages” of the technical reports contain material that is “exempt under subsection 45(1) of the FOI Act because disclosure would found an action, by a person (other than an agency or the Commonwealth), of a breach of confidence.” The primary concern in this case seemed to be that ACER might sue the department, but based on the Contract Notices available on AusTender covering the relevant period (CN3584909, CN3743777, CN3322415), it seems that there was no expectation of confidentiality between the parties. In the ‘Confidentiality – Contract’ and ‘Confidentiality – Outputs’ fields (more details on this here), the department put ‘No’, which apparently indicates that none of the following applied:
Additionally, in relation to 45(1), DESE state that “the analysis methodologies used to develop the data contained in technical reports can only be applied by a handful of specialist assessment organisations around the world” and that this “confers a competitive advantage on ACER.” As the department explained in their letter to me:
ACER is a private entity engaged by the department to administer the LANTITE and to collect and collate data for the purpose of internal analysis of LANTITE test participation and performance. … release of the data would reveal ACER’s methods of data analysis and collection. … disclosure of this material would have a detrimental effect on ACER’s commercial interests because the information would be advantageous to their competitors.
However, this claim is at odds with two other Contract Notices (CN3080212, CN3743777) which indicate that ACER’s services were procured through a limited tender process ‘due to an absence of competition for technical reasons.’ It seems to me that, when DESE wanted to justify awarding the contract to ACER without an open tender process, they have claimed that ACER has no competitors. On the other hand, when DESE want to justify refusing public access to the technical reports, they argue that ACER actually does have competitors.
Second, according to DESE, material contained on “certain pages” of documents 2 and 3 was “exempt under paragraph 47(1)(b) of the FOI Act” because it “represents substantial intellectual property of ACER with regard to test design and analysis, and disclosure of the information would reveal this expertise and the application of it in relation to the LANTITE to its competitors” and that “the psychometric processes and procedures set out in the documents form an integral part of ACER’s expertise in securing business in assessment development, administration and analysis.”
Presumably this “assessment development, administration and analysis” business includes the NAPLAN 2023 Technical Report, which, surprisingly, includes quite a lot of detail regarding ACER’s methods of test design and analysis. Page 46 of the NAPLAN 2023 Technical Report contains this passage describing ACER’s intellectual property, psychometric processes and procedures:
The results of this analysis are documented in Appendix B of the report, which stretches for 604 pages.
Perhaps ACER uses the same ConQuest software and Rasch scaling methodology for LANTITE. If this is the case, related information in documents 2 and 3 is already in the public domain. Or perhaps ACER uses a different methodology for LANTITE to that used for NAPLAN. In that case, it is unclear why information about ACER’s LANTITE methodology could not be shared publicly as has been done with their NAPLAN methodology.
Third, the material on “certain pages” was deemed
conditionally exempt under subsection 47F(1) of the FOI Act because release would involve the unreasonable disclosure of personal information about a person other than you and, pursuant to subsection of the FOI Act, is not required to be disclosed because disclosure at this time would be, on balance, contrary to the public interest.
I think DESE’s concerns here are legitimate. They’ve stated that the documents contain data about medical incidents which occurred during the test, and data which “relates to a very small number of individuals” who are reasonably identifiable based on demographic information also included in the documents. However, it seems to me that this material could be redacted. Further, this issue seems to have been addressed in the NAPLAN Technical Reports. I can’t see why these privacy concerns should mean that no technical information at all about LANTITE can be released.
Reviewing DESE’s decision
Once DESE informed me of their decision to refuse access to Documents 1, 2 and 3 (i.e., the 2019 Administration Report and Technical Reports for 2016 and 2017), my next step was to request an internal review. I submitted my request to DESE in February 2021 and, in March 2021, I was informed by an ‘Authorised FOI Decision-Maker’ that had taken the following into account while reviewing the primary decision:
The Authorised FOI Decision-Maker explained that they were “satisfied that the documents identified as exempt in the primary decision are exempt from disclosure” and had “decided to affirm the primary decision.”
Having received the outcome of the internal review, I requested an external review by the Australian Information Commissioner in March 2021. I argued that disclosure of the information contained in the reports is in the public interest. The relevant FOI Act subsection for making such a determination is 11B(3), which states that factors “favouring access to the document in the public interest include whether access to the document would … promote the objects of the Act (3 and 3A)” and “inform debate on a matter of public importance promote effective oversight of public expenditure.”
Regarding the first of these factors, although DESE has chosen to outsource its development and administration to ACER, LANTITE is a key aspect of Federal Government governance of Initial Teacher Education. As such, it is in the ‘Government processes’ and ‘activities’ referred to in Section 3(2) of the FOI Act. The information contained in Documents 1-3 is necessary for public participation in LANTITE-related ‘Government processes’ and “scrutiny, discussion, comment and review of the Government’s activities,” all of which will promote “better-informed decision-making” and, more broadly, Australia’s representative democracy.
Regarding the second factor, the quality of Initial Teacher Education students is clearly a high priority for DESE, the Federal Government and the Australian public more broadly. This is why LANTITE was introduced. Since its introduction, there has been public debate about LANTITE test results and their implications for our education system. The information contained in Documents 1-3 about the quality of the design, administration and analysis of LANTITE is quite clearly needed to inform this debate.
In relation to public expenditure, the Federal Government has paid over $2.5 million to ACER for the development, trialling and ongoing administration of LANTITE. Because the information contained in Documents 1-3 has not yet been released, the public is not able to evaluate the design, administration and analysis of LANTITE and, thus, “effective oversight of public expenditure” in relation to LANTITE is not currently possible.
What now?
After some time, the Office of the Australian Information Commissioner (OAIC) decided that my request for an external review had enough merit to appoint a review adviser. The OAIC then needed time to clear a huge backlog of applications that had accumulated during the transparency-averse Coalition governments between 2013 and 2021.
After a couple of years with no clear progress, I was finally contacted on October 11 2024 by an Assistant Review Adviser asking for submissions from me detailing why I contest the Department’s application that documents 1-3 are exempt from public disclosure because of sections 45, 47(1)(b), and 47F Personal privacy.
I have responded to the Assistant Review Adviser with my submissions and now the waiting continues. I’ll post again here when there are developments.
pedagogablog 
Positive to hear some progress here. Was glad to have received this blog post in my mailbox.