Educating All Parents To Ensure The Future Of Our Republic

School Board Challenge - Find just one valid independent study by 1/31/06

In early January I received an email from one of the Alpine School District Board members:

Oak, You've already got your mind made up but if by chance you are struck with a perfectly good moment of openmindedness, you could check out this website: The web site
Providing you with ALL information is helpful.
Why is it that you seem so destructive? I'm sure you will find teachers who have not appreciated the time it has taken to learn a new style of teaching math, but you will find no one who has been terminated because of it.
Every good teacher will take advantage of the option giving each student what they need as far as the math goes. We still believe that standard based math is the basis for the curriculum. By the way, the Governor wants, as part of his education agenda, a "deeper understanding of math." You can't get that with just drill. Happy new year to you! Sincerely, Name Withheld

Openmindedness has been the quest of this author from the very beginning. Less than a year ago I was neutral on Investigations math until I actually started doing research to know what the facts were for myself. So I gladly took up the opportunity to read about the COMAP study and then presented my findings at the January 2006 school board meeting.

Here's the glowing conclusion from the COMAP study:

“…The principal finding of the study is that the students in the NSF-funded reform curricula consistently outperformed the comparison students: All significant differences favored the reform students; no significant difference favored the comparison students. This result held across all tests, all grade levels, and all strands, regardless of SES and racial/ethnic identity. The data from this study show that these curricula improve student performance in all areas of elementary mathematics, including both basic skills and higher-level processes. Use of these curricula results in higher test scores.”

Unfortunately (for the board), the study had a tiny independence problem in that TERC (the group that created and publishes Investigations Math) was the financial backer of the group that created, administered, and published the study.

Dr. Jim Milgram of Stanford said of this study, it is akin to the tobacco company studies of the 50's and 60's telling everyone "smoking is safe, we've tested it."

In preparation for presenting the material to the board, I spent several hours researching and contacting people. One person I was directed to was Sandra Stotsky, who was the assistant commissioner of education in Massachusetts when this study was performed there in 2000. She wrote:

“I am aware of several major problems with the MA part of the study.  (1) As the Executive Summary admits, mostly high--income "white" schools were using the "reform" programs in the MA grade 4 sample, (2) no information is given on the supplemental tutoring that exists in these suburban communities (a hard factor to get information on without labor-intensive exploration at each school), (3) no information is given about supplemental curriculum materials the teachers themselves may have used--all we are told is that the schools that were contacted said they fully used the reform program.  I know that many teachers in these high-income schools use supplemental materials to make up for the "reform" programs deficiencies, (4) no information is given on the amount of professional development the "reform" teachers had (a huge amount in all probability) in comparison to the teachers in the comparison group (if no new math program, no professional development), (5) no information is given on the amount of time spent on math in the reform schools compared to the comparison group (the "reform" programs require a lot more time per week than most schools had been allotting math for many years.  For example, I discovered that one Newton elementary school with top scores was considered a model because it taught math one hour each day!), and probably most important and relevant (6) the MCAS grade 4 math test was originally designed with a great deal of advice from TERC.  TERC also shaped the math standards in the 1995 standards document that were being assessed by this test in 2000 (it is acknowledged in the intro to this document).  TERC's supporters (and EM supporters) were on the assessment advisory committees that made judgments about the test items and their weights for the math tests.  It is well-known that the grade 4 test reflects "constructivist" teaching of math.  In other words, the grade 4 test in MA in 2000 favored students using a "reform" program. “

I also contacted Dr. Jim Milgram at Stanford. He is on the advisory board of NASA and says that these programs have been around for decades and if they were effective, NASA, IBM and others would be actively looking for high schoolers that went through these programs.  He also said it's generally acknowledged that no valid study has ever been performed to show the effectiveness of these programs.  I will post his entire fascinating email at the bottom of this page, but I wanted to give you his quotes that I read to the board at the meeting from his review of Connected Math. I gave each member of the board a copy of this document and asked them to read it.

Page 1 of 22:If one visits the web site of the program (Connected Math),, one finds two preprints, presumably using rigorous methodology and statistical analysis, that are advertised as showing the benefits of CMP. Unfortunately, as we see in the appendix to this report, both studies are fatally flawed and deceptively presented. Additionally, at the website one will find a strong endorsement of the program by the AAAC. They grade it as one of the most effective programs for teaching middle school mathematics Unfortunately, this too must be taken with a grain of salt, as is also discussed in the appendix. In fact, it is generally acknowledged that there are no reputable studies showing that any of the NSF developed mathematics programs actually benefit students in testable ways.

Leaving aside these issues, we turn to the program itself.”

(Oak's comment: What a sense of humor he has!  Then he proceeds to closely examine the three grade levels for connected math and skillfully show this curriculum as one of the worst available.)

Page 2 of 22:Overall, the program seems to be very incomplete, and I would judge that it is aimed at underachieving students rather than normal or higher achieving students. In itself this is not a problem unless, as is the case, the program is advertised as being designed for all students. In fact, as indicated, there is no reputable research at all which supports this.”

Page 20 of 22: “There is a second paper extolling CMP at the CMP website by Reys, In this paper the statistics are done well but the "control group" is not realistic. The paper looks at three programs: CMP, another similar program, and a "control group" that consists of teachers who seem to share the same philosophy as the developers of CMP but are teaching without the assistance of any books or course materials. In other words the control group consists of teachers who are just winging it.

Unfortunately, this kind of statistical analysis, poorly done and misleading, appears to be very common in research on NSF funded programs, and the errors all seem to be in the direction most favorable to the programs.”

Besides presenting this material to the board, I challenged the board to 3 things:

1) Don't ever let someone from the district contradict me after I leave and then not tell me about it (which happened in either October or November) and not give me a chance to respond.

2) They've acknowledged that teachers are now able to teach anywhere on the spectrum that they think the students need to learn from Investigations to Traditional math, so now put your money where your mouth is and fund teacher choice of curriculum (the statement giving teachers authority to teach without fear of reprimand was read in November at the board meeting and the direct result of my October visit to the board).

3) A challenge to find JUST ONE VALID INDEPENDENT STUDY by the end of January that shows these constructivist programs are on par or better than traditional programs.  If they find one I agreed to publish it on my website and email it to all of you.  I didn't make them promise to drop the programs if they couldn't find one, but hopefully it will make some people think about what's happening as they try to find an untainted study.  Of course I'll keep you posted on this.

Below is the text of Dr. Milgram's email to me and then a final exchange with him.

Dr. James Milgram (PhD Math, Stanford University, Advisory Council for NASA):  There were very good reasons that responsible researchers at the US Department of Education, in their study of US middle school math programs, who were very aware of the ARC study you refer to below, graded it methodologically unsound - see the discussion on the What Works Clearinghouse page  The very first problem that I ran into was when I went to the ARC center website, I looked at the page entitled  "About the ARC Center"
What does one find?  The center is entirely supported by the three programs it is testing.  This matters because - one need only think of the tobacco studies of the 50's and 60's - one unfortunately cannot assume that research coming directly from parties with a financial interest in the outcomes is reliable.

Indeed, in the recent past I've looked closely at a number of reports by IMP and CorePlus, and have found, when looking at the way they crunched data, that the methods were deliberately skewed to get the results they wanted.

To give you an idea of the seriousness of the problem in the education research community, there was a recent paper by a researcher affiliated with a major school of education that I was asked to look at.  After a close study of the paper, I realized that the data had been essentially faked - this was a bit delicate, since the data quoted was mostly accurate, but the material was only selectively quoted - those parts of the data that could be interpreted as supporting the author's position - and that the tests constructed for the study had been deliberately written in such a way that the desired results would be achieved.  (It has been observed that the only thing education research has really shown over all these years is that students are likely to learn what they are taught and not learn what they are not taught, whether what they are taught is correct and useful or not.)  The university in question is currently taking account of my concerns and trying to deal with the situation.

The current reality is that, whatever the claims of improved test scores, the students coming out of our schools today simply do not match up to their competition.  I was just appointed to NASA's Advisory Council (with people like Neil Armstrong, etc.) because, as they told me when they invited me onto the board, NASA simply can't hire engineers and scientists today since they have to hire US citizens.  (My belief is, and it is supported by the work I did for the Smith-Richardson and Fordham Foundations on state math assessments, that the state assessments are so badly flawed that the numbers they give are essentially meaningless for any but the worst performing students.)

Companies like TI, INTEL, IBM have already made policy decisions to move all manufacturing and virtually all research out of the US (and out of Europe as well in some cases) and this is not to save money.  I'm sure you are aware of the fact that GM and Ford are on the verge of bankruptcy and are shutting down over 20 plants between them and letting go 30% of their US and Canadian workforces.  The root causes are bad financial decisions based on the belief that they could effectively compete with foreign products.  Again, it is not the costs so much as the fact that these foreign products are simply better.

The thing to keep in mind is that the programs you mention have been in place for a long time.  TERC, Everyday Math, etc. all date from the late 1980's and were being extensively used since the early '90's.  There has been plenty of time to see actual outcomes - young men and women in the workforce that employers are glad to hire, solid engineers, and competent young scientists.  They simply are not there.  I can't tell you the number of times I've heard your story.  A district naively goes "the reform" route, and then 5 or 6 years later, very privately drops those programs and tries to fix the resulting damage.

Fixing outcomes in mathematics is not that simple.  The reality is that the teachers K - 8 are functionally mathematically illiterate these days.  (Not all of them, of course, but the vast majority.)  What happens as a result is that they are not able to teach the subject reliably, regardless of the curriculum used.

Jim Milgram

(For those that frown on the Fordham Foundation, I asked Dr. Milgram why it is viewed negatively by some in the education establishment.  His reply was):

“The head of Fordham Foundation is Chester Finn, a former US Secretary of Education.  His focus then and now has been on trying to get good outcomes out of the educational process, as he is very much aware of the extreme urgency of the situation.  Yes, in a sense he can be regarded as extreme since he has been willing to criticize educational outcomes in this country, and the prevailing view currently seems to be ‘don't rock the money tree.’”

Jim Milgram

(Another follow up email to Dr. Milgram:
Oak: In your prior email you said:
 "the state assessments are so badly flawed that the numbers they give
 are essentially meaningless for any but the worst performing
 Without my own interpretation of this, will you please give me a short
 description of what you mean by only the worst performing students can
 have valid scores?


Dr. Milgram’s reply:

The average number of mathematically incorrect questions on these tests has been consistently in the 20 - 25% range - and this goes for NAEP as well.  With a situation like this, only the kids who know absolutely nothing and get very poor scores will be measured more or less accurately. Students in the mid to high ranges could either know the incorrect things that are wanted on the incorrect questions and have little idea of actual mathematical content, or they could know the math they should know well, and they will essentially be guessing on the incorrect questions.

Of course, the kids who get virtually all the questions "right" are a very interesting subgroup.  A few of them will actually know the math and understand the mindset that lies behind the incorrect questions as well. They could fully understand that they are giving incorrect answers to incorrect questions, but that the answers they give are what these silly adults want.  And yes, I can assure you that a few kids like that exist.

Of course, others in this group will have simply studied very hard and learned the correct responses to the given stimuli in all cases. Probably this is the largest subgroup of the high scorers, but such students have absolutely no idea of the actual mathematics they are supposed to be learning.

Jim Milgram


Investigations Math Menu

** Most important pages to read (all have value but if you will only read a few pages make it these)
* Very important

Research and Information Press Coverage

THE PETITION TO REMOVE INVESTIGATIONS MATH** (OVER 5% of ASD represented--still ignored by ASD)


Math With Madeline Video

All Weekly Email Updates **

Edspresso Article on WMD Comics

Weapons of Math Destruction Comics**

What Can You Do About Fuzzy Math in Your Area - 13 Ideas**

Brain Programming

ASD's Saxon Math Deception
Part 2 of the Deception

4/21/07 ASD Superintendent Calls Parents Extremists (for wanting the times tables in schools)

Alpine School District Residents Should Choose Saxon Math

9/13/06 Dual Enrollment Guidelines - Opting Out of Math or Other Classes

9/2/06 Jaime Escalante and NCTM Standards**

9/2/06 Amber Lee's Independent Research

The Math Story (ie. How I Got Started)

7/31/06 Educators Ignore Project Follow-Through**

7/26/06 Comparison of Alpine to Nebo and Provo Districts and comments on Orem City Breaking Off **

3/30/06 Utah Public Radio Broadcast with David Wright and Damon Bahr* (Right-Click and use Save As to download)

Insane Supplementary Materials Compound Investigations Math Nightmare**

Map of ASD Precincts and Election information- You must register to run by 3/17

3/2/06 David Wright's Mathematician Petition to the State of Utah*

2/14/06 Why are Charter School Scores Better than Alpine School District's?*

2/14/06 Alpine School District IOWA Test Math Scores

10 Myths About Math Education and Why You Shouldn't Believe Them

2/1/06 The Case for Utah adopting California's Math Standards-Testimony to Utah Congress**

1/10/06 School Board Meeting challenge to find just one valid study by 1/31/06 ** (I'm still waiting)

Math Tutors in Utah (find one/list one)

11/9/05 Update - How ASD is failing more children each year**

ASD's statement to teachers authorizing them to teach traditional math without fear of repercussion

California Study showing tripling of scores for schools that left constructivist math and implemented Saxon Math**

10/31/05 Petition Maps (see where people are signing the petition-blue dots are schools, red are petitions)

How About Grades 6-12 Connected and Interactive Math?**

Raising Critical Thinking Skills*

Pro-Investigations websites (for anyone interested)

8/15/05 Update (compares Alpine to other districts and charter schools) *

7/29/05 Update (resources given to ASD, sample problems, ASD vs. Nebo School District) *

Investigations Math Origins and Articles*

Math Programs Compared ("Traditional", Investigations, Saxon)

Supplementing your Public School Student's Education**

The California Investigations Math Failure**

Community comments - both Pro/Anti Investigations **

Investigations Math Poll Introduction and Question Summary*


Additional articles are covered in weekly updates since April 2007

Victory in Utah - State to Revamp Standards (11/16/06)

Major Breaking News...NCTM does a 180 after 17 years of destroying math in America

It must really irritate the folks at Alpine School District (ASD) who have taken to calling all my work "biased and flawed" to have such no-name disreputable organizations pandering to my lies. The legacy of ASD will be the disenfranchisement of tens of thousands of students unable to obtain a technical higher education degree.

4/25/07 Provo Daily Herald - Math petitioners called 'extremists' in Alpine District

4/24/07 Deseret News Article - So how many different programs are appropriate to use at one time Mr. Henshaw?

3/1/07 - Deseret News - Schools' grades mixed - Utah system rated by U.S. Chamber in a new report

2/14/07 Provo Daily Herald - Committee makes math program suggestions for Alpine: Many in district pine for 'Investigations'

2/14/07 Deseret News - Alpine seeking balance in math

2/12/07 Deseret News - Math Program Deleted - Alpine District dumps controversial approach

11/16/06 Deseret News - State leaders support math overhaul for schools

11/16/06 Salt Lake Tribune - Utah's 'fuzzy math' curriculum scrapped; legislators endorse plan to revamp state standards

11/15/06 New York Times - As Math Scores Lag, a New Push for the Basics

10/29/06 Provo Daily Herald - Survey: Math Program Doesn't Add Up

10/25/06 Deseret News - Students decline at 5 Utah colleges

10/19/06 Deseret News - Math learning is shallow in U.S., professor says

10/19/06 Deseret News - Math in Utah — 'fuzzy' or A-OK?

10/06 Salt Lake Tribune - Bad marks for Utah's schools

10/9/06 BYU NewsNet - Alpine School District Under Attack

9/30/06 Provo Daily Herald - Districts evaluate 'failures' in testing

9/29/06 Provo Daily Herald - Utah schools receive progress rankings

9/26/06 - Parents Know the Right Equation for Teaching Math - by Phyllis Schafly

9/12/06 Wall Street Journal - New Report Urges Return to Basics In Teaching Math

9/6/06 Provo Daily Herald - Parent suggests Alpine district be responsive

8/28/06 Provo Daily Herald - Alpine readies to let schools choose curriculum for math (be sure to check out the comments section)

8/26/06 Provo Daily Herald - Charter schools attract parents concerned about public education

7/29/06 Deseret News - Burned-out teachers problematic

7/26/06 Deseret News - Most favor creation of small school districts

5/28/06 Salt Lake Tribune - Alpine School District relents and will change its math offerings***

5/26/06 Deseret News - Alpine opts for choice in its math programs***

5/25/06 Provo Daily Herald - District Reviews Math Options***

5/23/06 - Provo Daily Herald -Orem OKs feasibility study of splitting from Alpine district

5/23/06-Deseret News - Orem not hastening into a schools split

5/23/06-Salt Lake Tribune Article-Orem to study Alpine District split

5/12/06 Provo Daily Herald - Math decision delayed

5/10/06 Provo Daily Herald - Petition Requests District Division

5/6/06 Deseret News - Utah to study states' math programs

4/16/06 Provo Daily Herald- Alpine District continues to lose students to charters

4/6/06 Provo Daily Herald- Charter school debate heats up in Alpine school district

3/20/06 Salt Lake Tribune - Alpine in grip of 'Math Wars'

3/3/06 Provo Daily Herald-Professor: Clearer math standards needed

3/2/06 Deseret News- Math Petition Circulating

2/19/06 San Jose Mercury News - Math back in forefront, but debate lingers on how to teach it.

2/18/06 Salt Lake Tribune-Battle over math teaching spreads

2/15/06 Salt Lake Tribune-Indifference to math skills part of the problem

2/15/06 Deseret News-Parents voice their views on Alpine math

2/15/06 Provo Daily Herald-Provo district seeks math balance

2/9/06 Deseret News - Alpine trio defend approach to math
Rebuttal Letter to Editor: Math approach is harmful

2/7/06 Deseret News - Alpine defends math classes

2/2/06 Deseret News - Lawmaker Gives Alpine Math an F

1/29/06 Salt Lake Tribune - Charter School Funding Tight

1/20/06 Deseret News Letter to Editor: Rote exercises pay vital role in mathematical education

1/7/06 Provo Daily Herald - Investigations: Not your parent's math

12/18/05 Provo Daily Herald -Math doesn't add up for all

12/13/05  15 Charter Schools Scheduled to Open Next Year

11/24/05 Provo Daily Herald - Parents mad over new math curriculum (be sure to read the comments)



Copyright 2005-14+, All Rights Reserved