Predictions are cheap. Every pitch deck, podcast and year-end LinkedIn post is stuffed with them. The rare move is the second half of the job: going back later, lining your call up against what happened, and printing the result where people can see it. That second half is where credibility gets manufactured, and Warren Buffett has been running the factory for sixty years. Here is how a founder can do the same, with numbers.
Q1. How good are professional forecasters, really?
- Philip Tetlock tracked 284 paid experts who made 82,361 forecasts between 1984 and 2003 (Tetlock, Expert Political Judgment, 2005; review in the International Journal of Forecasting). His famous summary: “the average expert was roughly as accurate as a dart-throwing chimpanzee.”
- CXO Advisory graded 6,582 public stock market calls from 68 market gurus. Aggregate accuracy came in at 46.9%, a hair worse than a coin. The best guru scored 68.2% and the worst scored 21.7% (CXO Advisory, Guru Grades; Rick Ferri, Forbes, 2013).
- Here’s the twist that matters for you. Forecasting is a learnable skill. In the Good Judgment Project, run for the U.S. intelligence research agency IARPA, the winning team beat the control crowd by 60% in year one and 78% in year two (Tetlock and Gardner, Superforecasting, via AI Impacts). So the field is crowded with guessers and nearly empty of people who can show a graded record. Scarce things get noticed.
Sources: CXO Advisory Guru Grades (left); Tetlock, Good Judgment Inc. and AI Impacts (right).
Q2. Why is Warren Buffett the patron saint of keeping score?
- The first page of every Berkshire Hathaway annual letter is a scoreboard: Berkshire’s yearly result against the S&P 500, row by row, going back to 1965. The bad years stay in. In 2025 Berkshire’s stock rose 10.9% while the S&P 500 returned 17.9%, and that loss sits right there on page one (Berkshire Hathaway 2025 Annual Report).
- Over the full run the table shows a compounded annual gain of 19.7% against 10.5% for the index, an overall gain of 6,099,294% against 46,061%. In plain dollars, $100 in 1965 grew to roughly $6.1 million versus about $46,000 in the index (Berkshire 2025 Annual Report; Visual Capitalist, March 2026).
- He grades his own mistakes out loud, too. From the 2024 letter: “During the 2019-23 period, I have used the words ‘mistake’ or ‘error’ 16 times in my letters to you.” In the same letter he wrote that “a ‘report’ is what a Berkshire CEO annually owes to owners” and warned that “if you start fooling your shareholders, you will soon believe your own baloney and be fooling yourself as well.”
Q3. What does a forecast with real stakes look like?
Buffett’s million-dollar bet is the cleanest template ever written. In Berkshire’s 2005 report he argued that professional money managers, as a group, would trail a cheap index fund. He then offered to wager $500,000 on it. Ted Seides of Protรฉgรฉ Partners took the other side, the pot grew to $1 million for charity, and the Long Now Foundation refereed it as Long Bets #362. The claim was painfully specific: over 2008 through 2017, “the S&P 500 will outperform a portfolio of funds of hedge funds, when performance is measured on a basis net of fees, costs and expenses.”
- He started out losing. In 2008 the index fund fell 37.0% while the five fund-of-funds lost between 16.5% and 30.1%. The index took the lead for the first time in year five (Long Now Foundation).
- The final score: index fund +125.8%, the five funds between +2.8% and +87.7% (Berkshire 2017 letter). Seides conceded early in a Bloomberg column: “for all intents and purposes, the bet is over. I lost” (Institutional Investor). Girls Inc. of Omaha collected the money.
- The recipe you can steal: a falsifiable claim, a deadline, a referee and a stake. At seed stage the stake can be a $100 donation. The structure does the persuading.
Sources: Berkshire 2025 Annual Report and Visual Capitalist (left); computed from the year-by-year table in Buffett’s 2017 letter (right). Fund D reported no 2017 figure.
Table 1. The final scorecard of Buffett’s bet (Berkshire Hathaway 2017 letter)
| Position | Total gain, 2008-2017 | Average annual gain |
|---|---|---|
| Fund-of-Funds A | 21.7% | 2.0% |
| Fund-of-Funds B | 42.3% | 3.6% |
| Fund-of-Funds C | 87.7% | 6.5% |
| Fund-of-Funds D | 2.8% | 0.3% |
| Fund-of-Funds E | 27.0% | 2.4% |
| S&P 500 index fund | 125.8% | 8.5% |
Q4. How do I build a simple, transparent model for my market?
- Pick one number people in your market argue about. How many Series A rounds close in your category next quarter. What a used EV battery pack sells for in December. What share of dental practices in your state adopt AI scheduling by year end. It must resolve on a known date from a public source, or it’s a vibe.
- Start with the base rate, then adjust. Bent Flyvbjerg’s database of more than 16,000 projects found 47.9% came in on budget, 8.5% on budget and on time, and just 0.5% on budget, on time and with the promised benefits (Flyvbjerg and Gardner, How Big Things Get Done, 2023; The Independent Review). Your gut says your plan is special. The reference class has seen that movie.
- Write down three to five drivers and weight them in a spreadsheet anyone can open. If a smart stranger can’t rebuild your number in fifteen minutes, the model is too clever to be trusted.
- Timestamp it in public. A blog post, a LinkedIn post or a GitHub commit works.
Q5. How do I state uncertainty without sounding like I’m hedging?
- Use a probability or an 80% range. “I’m 70% confident we’ll see more than 40 seed rounds in Q1” can be scored. “It could go either way” is fog with a microphone.
- Score yourself with the Brier score: the squared gap between your probability and what happened (1 if it happened, 0 if it didn’t). Zero is perfect. Saying 50% on every yes-or-no question earns 0.25 forever. A forecaster who says 80% and is right 80% of the time scores 0.16 (AI Impacts, using Good Judgment Project data; they quote 0.32 on the original 0-to-2 scale).
- Chart your calibration: bucket your calls by confidence and check the hit rate. Expect your 90% calls to disappoint you. Flyvbjerg calls optimism bias “perhaps the most prevalent” of the biases that wreck plans (Thought Economics interview).
Sources: Flyvbjerg and Gardner, How Big Things Get Done (left); hypothetical example of what to publish (right).
Q6. What should the published scorecard look like?
Keep it boring and complete. Every call, the confidence, the outcome, the score. Here’s an illustrative quarter for a hypothetical B2B founder:
Table 2. Example public scorecard (hypothetical numbers, real math)
| Forecast | Confidence | Outcome | Brier |
|---|---|---|---|
| Our vertical sees more than 40 seed rounds in Q1 | 70% | Yes (46) | 0.09 |
| Median contract price tops $18,000 | 60% | No ($16,900) | 0.36 |
| Two incumbents ship AI features by June | 85% | Yes | 0.02 |
| Segment churn stays under 3% a month | 80% | No (3.4%) | 0.64 |
| Category search volume up 25%+ year over year | 55% | Yes | 0.20 |
| Quarter average (coin flip = 0.25) | 0.26 |
Yes, this founder scored about the same as a coin. Publishing it anyway is the point, because next quarter’s number now has something to beat. Write one sentence on each miss, update the model, and keep the old versions online where people can compare them.
Q7. What does a public track record actually buy me?
- It separates skill from luck. In the Good Judgment Project, performance correlated 0.65 from one year to the next, and about 70% of superforecasters stayed in the top 2% the following year (AI Impacts). Without a record, a lucky streak and real skill look identical.
- It beats credentials. Good Judgment’s forecasters were more than 30% more accurate than U.S. intelligence analysts who had classified information (Good Judgment Inc.; David Ignatius, Washington Post). They had the news and a scorecard. The scorecard won.
- Small edges compound. A one-hour training module improved accuracy about 10%. Aaron Brown, then chief risk manager at AQR, put it this way: “It’s so hard to see (10%) because it’s not dramatic, but if it is sustained it’s the difference between a consistent winner who’s making a living, or the guy who’s going broke all the time” (Commoncog, quoting Superforecasting).
- It buys forgiveness. Buffett trailed the index by seven points in 2025 and printed it on page one without drama. Sixty years of rows had already earned him the benefit of the doubt.
Q8. What’s the 30-day starter kit?
- Week 1: choose three questions about your market that resolve within 90 days.
- Week 2: build the spreadsheet, base rate first, drivers second.
- Week 3: publish each call with a probability, a range and a resolution date.
- Week 4: put the resolution dates on your calendar and promise a scorecard post on each one, win or lose.
Buffett puts his on page one. You can start with a LinkedIn post and a spreadsheet. Either way, the receipts are what people remember.
Sources
- Berkshire Hathaway 2025 Annual Report: berkshirehathaway.com
- Berkshire Hathaway 2024 Annual Report (“mistake” count, “baloney” quote): berkshirehathaway.com
- Berkshire Hathaway 2017 letter (final bet scorecard): berkshirehathaway.com
- Long Now Foundation, Long Bets #362: longnow.org
- Institutional Investor, “Protรฉgรฉ Partners Pays Up in Buffett Bet”: institutionalinvestor.com
- Visual Capitalist, Buffett vs. the S&P 500: Growth of $100: visualcapitalist.com
- CXO Advisory, Guru Grades: cxoadvisory.com
- Rick Ferri, Forbes, “It’s Official! Gurus Can’t Accurately Predict Markets”: forbes.com
- Tschoegl and Armstrong, review of Tetlock’s Expert Political Judgment: faculty.wharton.upenn.edu
- Good Judgment Inc., The Superforecasters’ Track Record: goodjudgment.com
- AI Impacts, Evidence from the Good Judgment Project: aiimpacts.org
- Commoncog, “How the Superforecasters Do It”: commoncog.com
- Thought Economics, interview with Bent Flyvbjerg: thoughteconomics.com
- The Independent Review, review of How Big Things Get Done: independent.org
Discover more from 1000 Startups
Subscribe to get the latest posts sent to your email.
Claude Penland builds the marketing and business strategy for companies that are good at what they do and hard to find. Thirty years operating, one exit, eight of them as a practicing casualty actuary.
The free two-page read is genuinely free. Email claude@1000startups.com and I'll send back what I can see from the outside. Or see the work samples and how to work with me.