February 09, 2006

Effectiveness Monitoring

Monitoring is a process of collecting data for validation purposes. Implementation monitoring is checking to see that a project is done under the terms and conditions of a contract. Effectiveness monitoring is looking at the results of the project and determining whether they produced the desired effect. While the former gets lots of attention, the latter is disappearing completely with the tightening of budgets and focused attention on how implementation occurs.

To do proper effectiveness monitoring, one must understand the motives of the design of the original project. What was it that was trying to be accomplished. When a project is to stabilize a bank with rip-wrap, the gabian, rock wired together to created an anchor weight is the implemented item. If it meets specs, it gets signed off on and the contractor gets paid. If after this winters floods, that gabian is now centered in the river, the effectiveness of the project is zero. The rocks didn't stabilize the banks.

Nobody gets held accountable for the cost of the project - it can be written off as useless, but the price was settled and set via implementation contracts. As stewardship of public lands transfers managing capability once again to people who have no clue as to how to manage the resource, the monitoring of accountability has to be refocused on effectiveness rather than implementation. Best management practices BMP is a euphanism for whatever we currently happen to be doing that works.

If we are going to take this seriously with the purpose of getting things right, we have to stop jobbing the numbers to use science to prove agendas. The proper design of science has to be governed by reproducibility of data - not by artificial quality controls. While quality control should govern the methodology - properly trained data collectors, vetted in their equipment use, with access to experts to reason with them in their field of interpretation - the data collected should not even be looked at for anything more than technical accuracy until the experiment is complete. Statistics are applied to data, not quality control. Duplicated and Blanks and Splits are fine for vetting equipment, but never never never can be applied to field data during collection events. EPA and state DEQ equivalents take note - all your information on everything is worthless data - the system is jobbed from the onset.

But - that means that we have to go back and validate our fundamental assumptions of science once again. And while some people may scream at this monumental cost, give them the finger and insist that it is a necessary precondition for getting it right. I am not calling for reinventing the wheel. School children can reenact newton's apple - at the Safari one time a Douglas High School physics class used the bear tower to reenact the pendulum in Poe's The Pit and the Pendulum - and a black bear spent an hour watching from 10 feet away. Got up once to chase two other bears away, and came back and continued watching. We don't give animals near enough credit for their intellegiance and then are totally surprised when they show humanlike independence - even though we choose our pets because they like us, which helps us like us.

Anyway - i drifted - to take control (nasty word) of your own destiny - you have to understand the situation that you are surrounded by. If you only implementation monitor -you will see that you accomplished what you said you would do - you done the project - got paid - done deal. Whether the project worked or not is determine by effectiveness - how well you achieved your goal. That can only be evaluated over time, after the money is spent. The government is reprioritizing domestic funds and strengthening implementation requirements while defunding effectiveness monitoring. The current controversy with BLM suspending OSU funding over a peer-reviewed published piece of science that was not vetted through contracted channels, ** , begs the science question entirely.

If you wish to learn to measure, that is what science is about. Get a simple tool, a thermometer, and start measuring the air temperature at the same time every day. Write down the time and the day and the temperature. Do this for a whole week. Did the temperature for the week get warmer or cooler? How do you know? Oh - you measured it. Will the same thing happen next week? How can you find out? Guess what - we have been doing effectiveness monitoing of the assumption that it gets warmer as we go from winter to spring to summer. Would we have the same results if we started in September? How many times should we do this until the information is certain? Will there be weeks where the temperature goes down? How will that affect the trend? What trend? Oh - the trend demonstrated by the data that we collected for ourselves with our thermometer. Global warming?

howdt ta here DL

No comments: