Facebook Google Menu Linkedin lock Pinterest Search Twitter

Advertisement

policy

Oct 19, 2010

Insulation Fire Risk – The data is in

Back in February of this year when a debate popped up in the media over the insulation program – if one loosely defines “debate” as screeching “OMG!! YOUR HOUSES ARE ALL GOING T

User login status :

Share

Back in February of this year when a debate popped up in the media over the insulation program – if one loosely defines “debate” as screeching “OMG!! YOUR HOUSES ARE ALL GOING TO BURN DOWN”  – we thought that it might be worthwhile for someone to take their underpants off their head and have a squiz at what the data actually said.

What we found was that under every possible scenario, the government insulation program – far from increasing the rates of fire occurring from installing insulation – actually reduced the rate of fires and likely reduced the rate in a quite substantial manner.

Ultimately, the data strongly suggested that the insulation program actually made the industry safer in terms of fire risk. Some folks found that surprising since it went against the hysterics – but it’s only really surprising if you weren’t paying attention. The industry before the program was completely unregulated everywhere except in South Australia. As the program rolled out, increasing amounts of regulation aimed specifically at making the industry safer was implemented – purely in an attempt to manage some of the broad risk involved. So the initial result wasn’t particularly surprising at all when you look at the broad picture.

The original analysis was with preliminary information that was incomplete – but we attempted to control for a few issues to make a set of broad estimates that we thought would be relatively robust and accurate. Over the last week or so, the complete data has been released in various places and after crunching the numbers again with the complete data, we find that reality falls pretty much in the middle of our earlier estimates.

One of the problems we had in February was in trying to estimate the timeliness in the relationship between getting insulation installed and when a fire broke out as a result of that negligent  insulation going into your roof. If you had a dodgy installation, is it more likely that any fire would occur sooner rather than later, and if so, by how much?

That was the big question.

It’s also something we can start to answer.

What we need is the number of installations for each month of the program, as well as the number of fire incidents each month linked to the insulation program.

We can get a proxy for the monthly installation numbers from the  report of the Senate Environment, Communications and the Arts References Committee that looked into the Energy Efficient Homes package here on page 19 of the report (on page 13 of the pdf file).

It’s only a proxy because it doesn’t measure when the insulation was actually  installed, but when the money was claimed for the installation by the installer from the government.  There might be a few days or a week or so lag in these numbers – but that difference is ultimately meaningless anyway, as we shall see.

The other piece of data we need is the number of fire incidents reported by month. We can get that from the government’s Home Insulation Safety Plan website.

When we compare the two series, this is what we get – fire incidents on the left hand axis, installation claims on the right hand axis and the month on the bottom axis:

installation1

As we can see, it took a few months of growth in the installation numbers before we started to see a dramatic increase in the number of fire incidents. Similarly, when the program was stopped in February, it took about 6 months for the fire incidents to wash out of the system and return to normal.

What’s normal I hear you ask?

In previous years, around 85 fires per year occurred that were linked to insulation coming from about 65,000 installs of insulation every year. We know this because it was reported by the Department, the Hawke Report and the Australian National Audit Office report.

Over the total period from when the insulation program started until when the fire incident numbers (not the rate but the total numbers) reduced back to normal, there were 1.21 million installations completed and 197 fire incidents reported. Yesterday in Senate estimates, Martin Bowles,  Deputy Secretary of the Climate Change and Energy Efficiency under questioning from Liberal Party Senator Mary Jo Fisher, explained the numbers:

What we now know in relation to the program we have had 197 fires; 163 of those have been reported through fire brigades and the balance have been found through our inspection programs. We also know that of the 197 only 27 have caused structural damage. If we look at that in percentage terms of what was done within the program of 1.2 million homes, 197 fires roughly equates to less than 0.2 per cent—which is significantly less than what we understand to be the long-term average of those sorts of significant fire issues within insulation. If we look at structural damage, the 197 fire incidents are just that. Twenty-seven relate to structural damage of either the house or, in some cases, just the roof. It is quite insignificant in terms of the overall program but obviously significant for those involved.

So we know there was a direct relationship and a slight lag involved between installs and fires. We know that the lag effect has now washed out of the system (returning to normal fire incidence numbers) and we know that in previous years the numbers have been consistent from one year to the next, washing out the relevance of that lag in any arbitrary 12 month period. So we can compare the fire rates that occurred before the program in any given year to the full effects of what occurred under the program (over a longer 18 month time frame to bring in all possible lagged fire incidents into the program numbers)

installation2

Before the program, the industry experienced around 1 fire per every 765 odd installations – or 1.3 fires per every 1000 installs.

Under the insulation program, even though the number of total fires increased by 2.3 times, the number of installations increased by 18.7 times compared to what happened before the program. As a result, under the program the industry experienced 1 fire per every 6158 odd installs – or 0.16 fires per 1000 installs.

That makes the insulation program around 8 times safer in terms of fire incidents compared to the state of the industry before the program. Even if we take the best absolute possible estimates of what went on before the program – say, 80 fires per year off 75 thousand installs – the program is still 7 times safer in terms of fire incidents than what occurred before the program.

As we can see from the first chart – nearly all of the fires that could be caused by a negligent installation occur within the first 6 months of that installation. So we can’t see any significant evidence that fire incidents related to the insulation program are carrying over into periods beyond about 6 months after the actual installation itself.

This is something the Dept is also now across – as Dep Secretary said yesterday in estimates:

If we have a look at the fire data—we have seen that progress over the last couple of months—it tracks very closely with the installations and it has significantly reduced over the last few months. We know quite a bit about that data at the moment. We know that a lot of it tracks very closely with the installation. We know that things happened in the first short period of time after installation, and if you have a look at the statistics on the web you will see how that has tracked over time— down to a point where, from our understanding of the pre-home insulation rates and from what we have understood from the data that is out there, it is about two per 100 of the fires previous to the program actually related to an insulation issue.

It’s just a pity that everyone else wasn’t across it much, much earlier. So keep this in mind next time you see someone blowing journalistic junk out of their nether regions over fire risk and the insulation program.

The problems with the insulation program are much more complicated than simplistic twaddle over houses burning down. We might take a look at them a little later.

Possum Comitatus — Editor of Pollytics

Possum Comitatus

Editor of Pollytics

Political Commentator and Blogger

Get a free trial to post comments
More from Possum Comitatus

Advertisement

We recommend

From around the web

Powered by Taboola

98 comments

Leave a comment

98 thoughts on “Insulation Fire Risk – The data is in

  1. JP

    John64 @ 82:

    “Where 29% of 14,000 homes inspected up until March were found to have “some level of deficiency ranging from minor quality issues to serious safety concerns”.”

    windchange @ 90:
    “Context is everything. If I was in a facetious mood I would say, you know the observable, empirical reality of 4 deaths, 30 percent of installations with serious problems…”

    And so the hysterical rhetoric goes, as in one fell swoop each and every “minor quality issue” becomes a “serious problem” (plus 1% for good measure). Empirical reality my arse.

    windchange @ 91:
    “why on earth would you pick a stimulus measure that already has known risk factors (fires, metal staplers, foil etc ) so that even with significant improvements given its nationwide scale it will still result in much damage (small percentage * large number = significant damage) despite the benefits”.

    Because nothing is risk-free? Seriously, as someone who has installed both batts, foil, and foil-backed batts in my own house, I can vouch for the fact that level of expertise required is very, very low indeed, and as such it’s ideal for a stimulus program that needs to get going quickly. Much damage was NOT done. There were around 200 fires out of 1,200,000 installs, and most of those fires were small. Some workers were put at risk by their bosses, some tragically so, but that happens every day in the building trade, with scant regard for regulations. During the course of the insulation program, there were the usual 10000+ per annum house fires NOT caused by insulation. Whichever way you look at it, the risks were small, and as Possum pointed out, significantly smaller than they were before Garrett & Co got involved.

    I know about a dozen people who got insulation through this scheme. Out of those dozen, only one had any quality concerns, and he did what most homeowners would or should do: he checked the job on completion, and relayed his minor concerns about the workmanship of junior workers to the main contractor, who put things right the next day at no cost.

    I was discussing the idea that the scheme was a failure with him not so long ago, and he said that despite having to get the contractor back in, he regarded his job (and the program) as a huge success, and boasted of his lower energy costs. All the rhetoric in the world isn’t going to change the minds of people (and there are millions of them) who got a great deal from this rebate.

  2. windchange

    @Gaffhook

    Windchange, what came first? the Chicken (as in the shonky contractors who completely disregarded the regs as described in post 74) or the Egg (which is the auditors report).

    Gaff, I concede; your logic is irresistible. Removing the hook is indeed unsurpassed for an associated letting off.

    I’m a little disappointed however, you didn’t take me up on my challenge to truthify the auditor’s report so let’s do it together hey? I’ll do the first one and you can do the rest OK?


    As previously mentioned, HIP was terminated prematurely on 19 February 2010, because of ongoing safety and compliance concerns. (Pg 48)

    How about instead:

    As previously mentioned. HIP was terminated prematurely on 19 Feb 2010 because of the principle of Chicken-Egg. In HIP contexts, when installers are Murdoch-loving, right-leaning, ABC viewers, eggs will inevitably break; however, as with highways, sometimes omelettes result.

    Over to you for the following:

    If more stringent registration and training requirements had been in place from the start of the program, many of the subsequent quality and safety problems may also have been averted. (Pg 114)

    The report found that contributing factors to the recent errors were the scale of the programs being administered by the division with high volume transactions driven by demand, the speed of implementation and the early focus on stimulus. (Pg 85)

    In large measure, the focus by the department on the stimulus objective overrode risk management practices that should have been expected given the inherent program risks. (Pg 27)

    Of course I am selectively quoting here – there are other parts of the audit that point to some achievements in safety programs and further, that the flow of information to the Minister was compromised.

    My selective quoting however, I’d argue, is far more representative, than the impression created on these pages that: fire rates indicate a good program and/or the insanity of critics; that there were a few problems around the edges as with any program; that any activity is an exercise in risk. I get that.

    The truth is this was a God-awful program in conception and implementation. I defy anyone to read the auditor’s report and conclude otherwise.

    I’m outa here now but anyone still reading this exchange really should read the report to discover the truth. Crikey’s, Possum’s, BK’s analyses are usually pretty good and I assumed the same would apply to HIR. But dig a little deeper and Crikey’s reporting/commenting here is no better than The Australian’s bias in misrepresenting the Mckinsay Report on the NBN (and the BER report). Ironically, the former is probably a pushback against the latter.

    Crikey’s recent byline “Now with Extra Source” is apt but to spell it out they should really add

    “Because sometimes, with our correspondents, by Crikey you’re going to need it”

  3. windchange

    @Possum

    The origin of this famous quote is that *selected* use of statistics can be used to say anything. Your response is to say that I’m claiming that data can never be used to say anything? Odd.

    Context is everything. If I was in a facetious mood I would say, you know the observable, empirical reality of 4 deaths, 30 percent of installations with serious problems, a 0.5-1 billion remediation bill. etc. Etc.

    No I don’t think I am confusing them in my mind although I possibly did so in print. The “auditor’s installers” are the businesses, not necessarily the persons getting into roofs – say a rinstaller. The question here is: was it mandated that a rinstaller have some training prior to working? Yes the industry (with SA excepted) was unregulated but my reading of the report is that in every other rollout it was mandated that a rinstaller had to undergo prior training before stepping into potentially live roof cavities as “of course” they should have been.

    I accept that many improvements were made to training packages in HIS and that it is likely that some of these contributed to the reduced fire risk. But what is the point in having improved training packages if they are not mandated (or mandated at the eleventh hour) for rinstallers that is. Well, to put in bluntly, the point was stimulus was far more important than safety than it should have been. This is an example of my policy analysis.

    I accept that the fire data is being used in questionable reasoning (* although see below). But the use of faulty reasoning to condemn a shocking program doesn’t render the program any less shocking. You accuse me of a lack of policy analysis but all I was doing was calling your selected use of statistics – to criticise fire-fixation as a poor substitute for policy analysis – as itself, an equally poor substitute for policy analysis.

    My attitude to the HIS originally came from an instinctive reaction that well that seems like a bit nutty (yes easy in hindsight), and yes was probably influenced by the Australian coverage but it then hardened when seeing the 4-corners program. Then I read the Auditor’s report and couldn’t believe it. It’s like watching an unfolding car crash, maybe as BK said all auditor’s reports are like this – its what they do – but I can’t help a gnawing feeling- the blood of those kids are on our hands. How anyone could use this report to fixate on fire risk beats me.

    Yes there are many questions about policy implementations. To your points: of course you don’t treat businesses and household agents as rational or even ethical agents (yes it to the degree that they are rational or eithical that is important). When weighing up conflicting advice look at the self-interest (installers would say that rinstaller training is unnecessary wouldn’t they?), try and include all stakeholders (e.g. Consulting sparkies on risks to rinstallers in live roofs might be a good idea), learn from previous implementations (metal staples, foil the effect of timelines, NZ, NK), and listen and act on advice when problems arise (how much stimulus is a life worth?) Aren’t all these kind of self-evident? Maybe all politicians should have the training a Politics 101 course?
    I agree a risk modelling group within treasury a good idea. I would go further and say it needs all the scientific tools of big data at its disposal, hooked up to the NBN (as part of gathering data for the models) and should be published online with sliders, buttons, mathematical equations, and assumptions listed for the public to assess exactly what and how the risks are being taking.

    *And finally, going to John’s earlier point about risk and perhaps where the fire-fixation starts – why on earth would you pick a stimulus measure that already has known risk factors (fires, metal staplers, foil etc ) so that even with significant improvements given its nationwide scale it will still result in much damage (<> large number = significant damage) despite the benefits.

    If you must stimulate then surely there are stimulus measures with lower risk profiles (NBN rollout anyone?) And if there are not, then I question whether 10K jobs – most of which have now evaporated – as per the report- and difficult to measure environmental and economic benefits – again as per the report – have been worth the original risk profile – imho this is what a more complete evaluation of data would have concluded.

  4. Possum Comitatus

    Windchange went:

    [If ever there is a textbook case of lies, damn lies, and Crikey Statistics, it is this article and the notion that a rate reduction from 2.3 vs 0.16 implies program success.]

    What the data says – you know, that thing called observable, empirical reality that we can measure (no idea what your preferred alternative is) – is that fire rates decreased with the program.

    Far from this being unexpected, we would actually expect such a thing to occur as the industry moved from a completely unregulated one (apart from SA) before the program, into one where regulations were initially imposed and continued to increase throughout the program, where higher standards were introduced both in terms of training requirements of installers and product eligibility *as* the program rolled out.

    You seem to be confusing installer registration (the installer that owns the business, registering to undertake work for the program) with the actual installation workforce (all of the people that actually do the installation)

    Before the program – there was no training requirements. None – just the usual, stock standard OH&S requirements.

    When the program started, a set of weak minimum standards for the workforce were introduced, where, if people working to install insulation didn’t have one of the three minimum competency requirements (prior industry experience; a qualification in an approved trade; or insulation specific training), they had to be supervised by someone that had.

    In December 09, that minimum competency requirement was expanded to include all people working in the installation side of the industry, not just supervisors and actual registered installers.

    As a result, with safety requirements on both the installer workforce and product use being tighter through the program than what existed before HIP, we should have expected to see a reduction in fire rates.

    On just this fire risk issue, the question here is “did the program reduce fire risk enough?”. And that question goes to a much larger debate about what we, as a country, are willing to define as acceptable risk in public policy. For instance, using a much more controversial example, we accept that construction deaths are an inevitable consequence of the construction industry itself – but we don’t refuse to build highways because of it. We deal with the issue through a prism of acceptable risk.

    But rather than that question – and it’s important flow on issues – the debate has been dominated by the lie that “fire risk increased”.

    I don’t think you’ll find anyone here suggest that there were no mistakes with the program – there were a great many. But as to what those mistakes actually were, you wont read about it in the media because nonsense like “your houses are going to burn down!” became a substitute for actual policy analyis. Similarly, most people – like you are doing here – have little to say about it either, getting caught up on what are really the implications of your own imagination.

    How do departments deal with program design when industry advice coming to them is highly conflicted?

    Is third party risk assessment for program implementation a worthwhile endeavor when it so often struggles with the unique issues of public policy risk (as it did here), or do we really need to start thinking about creating a specific risk modelling group within Treasure or Finance that is filled with serious corporate knowledge on public policy design and management? This comes right down to the mechanics of whether policy expectations are unrealistic because of, say, time constraints and the process of how they could be adapted or changed to meet the time criteria.

    Should we treat small business and households as rational agents in public policy design when this program clearly demonstrates they were both far from rational – which has pretty fundamental consequences for policy design in terms of the trade-offs between increased program cost and reduced program risk.

    With demand led programs, how do we deal with the trade-off between reduced program uptake and either increased upfront private costs on the one hand, or the size of punitive punishments for breaches of the terms and conditions attached to the actual service delivery on the other hand?

    They’re a few of the big issues to come out of the program and it’s reviews (both Hawke and ANAO) – let alone the process by which Departmental executive provides information to the Minister.

    Fire risk? Not only was it piddling, it was substantially reduced from a level that we all previously accepted anyway.

  5. windchange

    @John64, If ever there is a textbook case of lies, damn lies, and Crikey Statistics, it is this article and the notion that a rate reduction from 2.3 vs 0.16 implies program success. If Crikey was your only source you could be forgiven for thinking that the pink batts was basically a success story and that any intimations to the contrary – a Murdoch/ABC right-wing hatchet job.

    Many here have tried to claim that appropriate regulations were in place (thereby attempting to entirely shift culpability from the government to unscrupulous installers and by some nasty posters, to the boy’s themselves). Well, it is hard to imagine a single paragraph more damning (pg 53) from an Auditor:

    Individuals new to the industry could participate in the program 
    without any experience, qualifications or insulation specific training, as the 
    minimum  competency  requirements  did  not  become  mandatory  for  all 
    installers  until  12 February  2010,  immediately  prior  to  program  closure. 

    The Auditor’s conclusion a few sentences later …

    … Consequently,  the  installer  registration  process  failed  to 
    provide a satisfactory level of assurance as to the competence of installers or to 
    the quality and safety of installations undertaken. 

    A less anodyne conclusion might be – No sh*%*t.

    Every other noted roll-out has required mandatory training for all installers (not just supervisors) from the UK, to NZ to previous ones here in Australia (Victoria).

    There was an appalling inevitability about the unfolding of these events. The very moment those regulations were put in place there was a miserable roof somewhere in the land waiting to fill the consciousness of some poor kid’s last moment.

    As the report makes clear over and over, the urgency of rolling out the program trumped everything else. Efficiency and safety concerns ran a poor second. We simply had to “go early, go hard and go household”.

    Don’t take my word for it – anyone seriously curious or masochistic- read the auditor’s report:

    http://www.anao.gov.au/download.cfm?item_id=1E0F5BC31560A6E8AA7094F5991383A7&binary_id=A29F35F31560A6E8AA1C1856F3A826C4

  6. John64

    @autocrat & oldskool: The Auditor-General disagrees with you.

    “The focus by the department on the stimulus objective overrode risk management practices that should have been expected given the inherent program risks”.

    “The department intended to rely heavily on its compliance and audit program to address some of the risks identified, but the significant delay in implementing this element of the program meant that these risks were not adequately addressed.”

    Mr McPhee says that by November 2009 the number of complaints to the department over quality, safety and fraud overwhelmed the department and as issues requiring attention increased, the department could not deal with the volume.

    The report estimates safety inspections and the removal of foil insulation from around 50,000 homes could cost around $85 million and ongoing inspections for another 150,000 homes could cost $340 million.

    A program that:
    – Requires 200,000 inspections.
    – Where 29% of 14,000 homes inspected up until March were found to have “some level of deficiency ranging from minor quality issues to serious safety concerns”.
    – 4,000 potential cases of fraud associated with the program.
    – Estimated $500 Million to fix the faults discovered so far… for a program that cost $1 Billion.

    This was by no means a “successful” program and it is clear, very clear, that known risks were: over-ridden, systems to address them were not implemented, they not adequately addressed and the Department was over-whelmed by the volume of complaints. They’re all nice ways of saying nothing was done. Complaints were left ignored, on a pile, unaddressed and ultimately unanswered in a letter sitting in Rudd’s office.

    But no, tell me more about how fantastic this program was guys and keep scratching your heads wondering why the Government doesn’t defend itself on this one (Hint: They’d have to be completely delusional to do so).

  7. John64

    I’m constantly surprised by the Apologist Squad when it comes to the home insulation program. People are actually asking why the Government doesn’t defend these numbers more? Well, here’s the problem with that…

    First, let’s start with the numbers, courtesy of Possum. Under the insulation program:
    – 1.21 million installations were completed.
    – the industry experienced 1 fire per every 6158 odd installs (0.16 fires per 1000 installs).
    – Before the program, we had 1 fire for every 765 installations (1.3 fires per every 1000 installs).

    1.21 M / 765 = 1,581 fires that would have been expected before the program started.

    Now let’s go back in time to Program Announcement Day #1. Minister Garrett is making his announcement and it goes a little something like this…

    “Under this program the Government will install insulation in approximately 1.21 million homes. Beyond reducing energy consumption by a small margin, we also expect this program will result in 1,500 house fires ($X million damage? What is the cost of an average house fire these days?) and cause 10 – 12 deaths. If you’d like your home to catch fire or one of those deaths to occur in your roof, sign up today!

    The industry has also been in touch with us from the very outset warning us of significant problems which will cause these deaths and that most of them could be easily avoided with changes to the program, changes to regulation dealing with materials used in insulation or better training programs. We currently do not intend to implement any of these at this stage as we find 12 deaths and 1,500 house fires to be an “acceptable risk”.

    Yes, your argument is that the Government was aware of the significant dangers involved from day one (I consider being aware of any program that may cause a death as a “significant” danger – ref history of the construction industry), that the Government knew people would die and that they knew houses would catch fire as a direct result of this program. And rather than do something about it, they chose to do nothing.

    It really scares me to see the sort of Government some of the people here actually want. “I really do want my Government to burn down homes and kill people. It’s acceptable to me!”

    This would be like a State Government not caring about a death caused by a dangerous road because the road toll is still lower than last year. It would be like 20 soldiers dying in a war and no-one giving a damn because it’s “statistically acceptable” – even if it’s found those deaths could have been avoided with better training, more equipment, more troops on the ground or any other possible resolution.

    … and you wonder why the Government doesn’t come out and say “Oh, this is all ok – because we expected it to be much worse!!”

  8. Gaffhook

    Windchange.

    If all the shonks/honest employers and anyone associated with the lieberals bothered to conduct their buisness in a proper manner and would be aware of the following which, although this is the Qld version, virtually is the same in all states.

    Read it and if you care go to the link and read the lot and then STFU.

    This is what they are obliged to do before they start work on the site.

    There would have been no accidents if the greedy, shonky, shortcutting fraud merchants had have complied with this.

    [2. What is risk management?

    The risk management process required by section 27A of the act is systematically divided into five steps:
    1.
    identify hazards, based on experience, recorded data and other information
    2.
    assess the associated risks by making an evaluation of the level of risks to the health and safety of workers, based on the consequences and likelihood of harm
    3.
    select control measures from the hierarchy of control (e.g. eliminate, substitute, isolate or engineer out the risks, or reduce them through administrative measures or personal protective equipment) by selecting the highest order control method possible and then proceeding down the list in order
    4.
    implement or apply the selected control measure(s) in the workplace
    5.
    monitor the control measures to ensure that they are working correctly to control the risks and that no other risks have been introduced.
    Effective risk management involves identifying all of the hazards in the workplace, and then carrying out a risk assessment for each hazard, to assess the severity of a risk, before deciding its priority.
    When carrying out a risk assessment, determine the risks that have the greatest potential to cause harm and a greater likelihood of occurring. These risks are controlled first, followed by the less serious risks.
    Attention should be given to risks that may be easy to fix but may have low risk priority scores (e.g. power leads across the floor). These risks should be fixed promptly. Particular attention should be given to risks that may have very low likelihood of causing harm but may result in major consequences.]

    http://www.deir.qld.gov.au/workplace/resources/pdfs/riskman_code2007.pdf

Leave a comment