Thank you for tuning in to this audio only podcast presentation. This is week 136 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Econometric election models.”

It has been a few weeks here since we started by digging into a good Google Scholar search and you know this topic would be just the thing to help open that door [1]. My searches for academic articles are always about finding accessible literature that sits outside paywalls that is intended to be read and shared beyond strictly academic use. Sometimes that is easier than others when the topics lend themselves to active use cases instead of purely theoretical research. Most of the time these searches to find out what is happening at the edge of what is possible involve applied research. Yes, that type of reasoning would place me squarely in the pracademic camp of intellectual inquiry. 

That brief chautauqua aside, my curiosity here is how do we build out econometric election models or other model inputs to feed into large language model chat systems as prompt engineering for the purposes of training them to help either predict elections or interpret and execute the models. This could be a method for introducing extensibility or at least the application of targeted model effect to seed a potential future methodology within the prompt engineering space. As reasoning engines go it’s possible that an econometric frame could be an interesting proxy model within generative AI prompting. It’s a space worth understanding a little bit more for sure as we approach the 2024 presidential election cycle. 

I’m working on that type of effort here as we dig into econometric election models. My hypothesis here is that you can write out what you want to explain in a longer form as a potential input prompt to train a large language model. Maybe a more direct way of saying that is we are building a constitution for the model based on models and potentially proxy models then working toward extensibility and agency from introducing those models together. For me that is a very interesting space to begin to open up and kick the tires on in the next 6 months. 

Here are 6 papers from that Google Scholar search that I thought were interesting:

Mullainathan, S., & Spiess, J. (2017). Machine learning: an applied econometric approach. Journal of Economic Perspectives, 31(2), 87-106. https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.31.2.87 

Fair, R. C. (1996). Econometrics and presidential elections. Journal of Economic Perspectives, 10(3), 89-102. https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.10.3.89

Armstrong, J. S., & Graefe, A. (2011). Predicting elections from biographical information about candidates: A test of the index method. Journal of Business Research, 64(7), 699-706. https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/PollyBio58.pdf 

Graefe, A., Green, K. C., & Armstrong, J. S. (2019). Accuracy gains from conservative forecasting: Tests using variations of 19 econometric models to predict 154 elections in 10 countries. Plos one, 14(1), e0209850. https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0209850&type=printable

Leigh, A., & Wolfers, J. (2006). Competing approaches to forecasting elections: Economic models, opinion polling and prediction markets. Economic Record, 82(258), 325-340. https://www.nber.org/system/files/working_papers/w12053/w12053.pdf 

Benjamin, D. J., & Shapiro, J. M. (2009). Thin-slice forecasts of gubernatorial elections. The review of economics and statistics, 91(3), 523-536. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2860970/pdf/nihms190094.pdf 

Beyond those papers, I read some slides from Hal Varian on “Machine Learning and Econometrics” from January of 2014 [2]. The focus of the slide was applied to modeling human choices. Some time was spent on trying to understand the premise that the field of machine learning could benefit from econometrics. To be fair since that 2014 set of slides you don’t hear people in the machine learning space mention econometrics that often. Most people talk about Bayesian related arguments. 

On a totally separate note for this week I was really into running some of the Meta AI Llama models on my desktop locally [3]. You could go out and read about the new Code Llama which is an interesting model trained and focused on coding [4]. A ton of researchers got together and wrote a paper about this new model called, “Code Llama: Open Foundation Models for Code” [5]. That 47 page missive was shared back on August 24, 2023, and people have already started to build alternative models. It’s an interesting world in the wild wild west of generative AI these days. I really did install LM Studio on my Windows workstation and run the 7 billion parameter version of Code Llama to kick the tires [6]. It’s amazing that a model like that can run locally and that you can interact with it locally using your own high end graphics card.

Footnotes:

[1] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=econometric+election+prediction+models&btnG= 

[2] https://web.stanford.edu/class/ee380/Abstracts/140129-slides-Machine-Learning-and-Econometrics.pdf 

[3] https://ai.meta.com/llama/ 

[4] https://about.fb.com/news/2023/08/code-llama-ai-for-coding/ 

[5] https://ai.meta.com/research/publications/code-llama-open-foundation-models-for-code/

[6] https://lmstudio.ai/

What’s next for The Lindahl Letter? 

• Week 137: Tracking political registrations
• Week 138: Prediction markets & Time-series analysis
• Week 139: Machine learning election models
• Week 140: Proxy models
• Week 141: Expert opinions

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

Thank you for tuning in to this audio only podcast presentation. This is week 135 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Polling aggregation models.”

I read and really enjoyed the book by Nate Silver from 2012 about predictions. It’s still on my bookshelf. Strangely enough the cover has faded more than any other book on the shelf. 

Silver, N. (2012). The signal and the noise: Why so many predictions fail-but some don’t. Penguin.

That book from Nate is sitting just a few books over from Armstrong’s principles of forecasting. A book that I have referenced a number of times before. It will probably be referenced more as we move ahead as well. It’s a resource that just keeps on giving. Math it’s funny like that. 

Armstrong, J. S. (Ed.). (2001). Principles of forecasting: a handbook for researchers and practitioners (Vol. 30). Boston, MA: Kluwer Academic.

My podcast feed for years has included the 538 podcast where I listened to Nate and Galen talk about good and bad uses of polling [1]. Sadly, it does not currently feature Nate after the recent changes over at 538. They reported on and ranked a lot of polling within the 538 ecosystem of content. Model talk and the good or bad use of polling were staples in the weekly pod journey. I really thought at some point they would take all of that knowledge about reviewing, rating, and offering critiques of polling to do some actual polling. Instead they mostly offered polling aggregation which is what we are going to talk about today. On the website they did it really well and the infographics they built are very compelling. 

Today setting up and running a polling organization is different from before. A single person could run a large amount of it thanks to the automation that now exists. An organization with funding to set up automation and run the polling using an IVR and some type of dialogue flow [2]. Seriously, you could build a bot setup that placed calls to people and completed a survey in a very conversational way. That still runs into the same problem that phone survey methods are going to face. I screen out all non-contact phone calls and I’m not the only person doing that. Cold calls are just not effective for business or polling in 2023 and the rise of phone assistants that can effectively block out noise are going to make the phone methodology even harder to effectively utilize.

It’s hard to make a hype based drum roll on the written page. You are going to have to imagine it for me to get ready for this next sentence. Now that you are imagining that drum roll… Get ready for a year of people talking about AI and the 2024 election. It probably won’t get crypto bad in terms of the hype trane showing up to nowhere, but it will get loud. I’m going to contribute to that dialogue, but hopefully in the softest possible way. Yeah, I’m walking right into that by reflecting on the outcome of my actions while simultaneously writing about them during this missive.

You can see an article from way back in November 2020 talking about how AI does show some potential to gauge voter sentiment [3]. That was before all of the generative AI and agent hype started. Things are changing rapidly in that space and I’m super curious about what can actually be accomplished in that space. I’m spending time every day learning about this and working on figuring out ways to implement this before the next major presidential election in 2024. An article from The Atlantic caught my attention as it talked about how nobody responds to polls anymore and started to dig into what AI could possibly do in that space, microtargeting, and Kennedy (1960) campaign references [4]. That was an interesting read for sure but you could veer over to VentureBeat to read about how AI fared against regular pollsters in the 2020 election [5]. That article offered a few names to watch out for and dig into a little more including KCore Analytics, expert.ai, and Polly. 

We will see massive numbers of groups purporting to use AI in the next election cycle. Even The Brooking Institute has started to share some thoughts on how AI will transform the next presidential election [6]. Sure you could read something from Scientific American where people are predicting that AI could take over and undermine democracy [7]. Dire predictions abound and those will probably also accelerate as the AI hype train pulls up to election station during the 2024 election cycle [8][9]. Some of that new technology is even being deployed into nonprofits to help track voters at the polls [10].

Footnotes:

[1] https://projects.fivethirtyeight.com/polls/ 

[2] https://cloud.google.com/contact-center/ccai-platform/docs/Surveys 

[3] https://www.wsj.com/articles/artificial-intelligence-shows-potential-to-gauge-voter-sentiment-11604704009

[4] https://www.theatlantic.com/technology/archive/2023/04/polls-data-ai-chatbots-us-politics/673610/ 

[5] https://venturebeat.com/ai/how-ai-predictions-fared-against-pollsters-in-the-2020-u-s-election/

[6] https://www.brookings.edu/articles/how-ai-will-transform-the-2024-elections/ 

[7] https://www.scientificamerican.com/article/how-ai-could-take-over-elections-and-undermine-democracy/

[8] https://www.govtech.com/elections/ais-election-impact-could-be-huge-for-those-in-the-know

[9] https://apnews.com/article/artificial-intelligence-misinformation-deepfakes-2024-election-trump-59fb51002661ac5290089060b3ae39a0 

What’s next for The Lindahl Letter? 

• Week 136: Econometric election models
• Week 137: Tracking political registrations
• Week 138: Prediction markets & Time-series analysis
• Week 139: Machine learning election models
• Week 140: Proxy models

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

Thank you for tuning in to this audio only podcast presentation. This is week 134 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “The chalk model for predicting elections.”

Last week we started to mess around with some methods of doing sentiment analysis and setting up some frameworks to work on that type of effort. This week we take a little different approach and are going to look at an election model. I’m actively working on election focused prompt based training for large language models for better predictions. Right now I have access to Bard, ChatGPT, and Llama 2 to complete that training. Completing that type of training requires feeding election models in written form as a prompt for replication. I have been including the source data and written out logic as a part of the prompt as well.

Party registration drives the signal. Everything else is noise. That is what I expected to see within this model. It was the headline that could have been, but sadly could not be written. It turns out that this hypothesis could be tested. You can pretty easily try to view the results as a March Madness college basketball style bracket. Accepting that chalk happens or to be put more bluntly the higher ranked seeds normally win. Within the NCAA tournament things are more sporting and sometimes major upsets occur. Brackets are always getting busted. That is probably why they have ended up branding it as March Madness. Partisan politics are very different in terms of the chalk being a lot more consistent. Sentiment can change over time and sometimes voter registration does not accurately predict the outcome.

We are going to move into the hypothesis testing part of the process. This model accepts a bi-model two party representation of political parties with an assumption that generally the other parties are irrelevant to predicting the outcome. The chalk model for predicting elections based on registration reads like this, the predicted winner = max{D,R} where D = registered democrats and R = registered republicans at the time of election. For example, the State of Colorado in December of 2020 that would equate to the max{1127654,1025921} where registered Democrats outnumber registered Republicans [1]. This equation accurately predicted the results of the State of Colorado during the 2020 presidential election. 30 states report voter statistics by party with accessible 2020 archives. Using the power of hindsight we can test the chalk model for predicting elections against the results of the 2020 presidential elections. 

Several internet searches were performed using Google with the search, “(state name) voter registration by party 2020.” Links to the referenced data are provided for replication and or verification of the data. Be prepared to spend a little time completing a verification effort as searching out the registered voter metric for each of the states took about 3 hours of total effort. It will go much faster if you use the links compared to redoing the search from scratch. Data from November of 2020 was selected when possible. Outside of that the best fit of the data being offered was used. 

1. Alaska max{78664,142266}, predicted R victory accurately [2]
2. Arizona max{128453,120824}, predicted D victory accurately [3]
3. California max{10170317,5334323}, predicted D victory accurately [5]
4. Colorado max{1127654,1025921}, predicted D victory accurately [6]
5. Connecticut max{850083,480033}, predicted D victory accurately [7]
6. Delaware max{353659,206526}, predicted D victory accurately [8]
7. Florida max{5315954,5218739}, predicted D victory in error [9] * The data here might have been lagging to actual by 2021 it would have been accurate at max{5080697,5123799}, predicting R victory
8. Idaho max{141842,532049}, predicted R victory accurately [10]
9. Iowa max{699001,719591}, predicted R victory accurately [11]
10. Kansas max{523317,883988}, predicted R victory accurately [12]
11. Kentucky max{1670574,1578612}, predicted D victory in error [13] * The data here might have been lagging to actual voter sentiment. The June 2023 numbers flipped max{1529360,1593476}
12. Louisiana max{1257863,1020085}, predicted D victory in error [14,15]
13. Maine max{405087,321935}, predicted D victory accurately [16]
14. Maryland max{2294757,1033832}, predicted D victory accurately [17]
15. Massachusetts max{1534549,476480}, predicted D victory accurately [18]
16. Nebraska max{370494,606759}, predicted R victory accurately [19]
17. Nevada max{689025,448083}, predicted D victory accurately [20]
18. New Hampshire max{347828,333165}, predicted D victory accurately [21]
19. New Jersey max{2524164,1445074}, predicted D victory accurately [22]
20. New Mexico max{611464,425616}, predicted D victory accurately [23]
21. New York max{6811659,2965451}, predicted D victory accurately [24]
22. North Carolina max{2627171,2237936}, predicted D victory in error [25,26]
23. Oklahoma max{750669,1129771}, predicted R victory accurately [27]
24. Oregon max{1043175,750718}, predicted D victory accurately [28]
25. Pennsylvania max{4228888,3543070}, predicted D victory accurately [29]
26. Rhode Island max{327791,105780}, predicted D victory accurately [30]
27. South Dakota max{158829,277788}, predicted R victory accurately [31]
28. Utah max{250757,882172}, predicted R victory accurately [32]
29. West Virginia max{480786,415357}, predicted D victory in error [33]
30. Wyoming max{48067,184698}, predicted R victory accurately [34]

This model predicting a winner with the max(D,R) ended up with incorrect prediction outcomes in 5 states during the 2020 presidential election cycle including Florida, Kentucky, Louisiana, North Carolina, and West Virginia. All 5 of these states based on voter registration data should have yielded D victory, but did not perform that way in practice. Some of these states clearly have shifted voter registration and I have added some notes to show those changes in Kentucky and Florida. It is possible that in both of those states voter registration was a lagging indicator compared to the sentiment of votes cast. The chalk model for predicting elections ended up being 25/30 or 73.52% accurate. 

You can imagine that I was expecting to see a much more accurate prediction of elections out of this chalk model. Again, calling back to that March Madness and thinking about what it means to have a clear path to victory for registered voters, but it not working out that way. So, that is why we tested this hypothesis of the chalk model. You can obviously see here that it is accurate most of the time, but not all the time. It’s something that we will continue to dig into as I look at some other models and I do some other tests with voter data while we are looking at elections and how they intersect with AI/ML.

Footnotes:

[1] https://www.sos.state.co.us/pubs/elections/VoterRegNumbers/2020/December/VotersByPartyStatus.pdf or https://www.sos.state.co.us/pubs/elections/VoterRegNumbers/2020VoterRegNumbers.html 

[2] https://www.elections.alaska.gov/statistics/2020/SEP/VOTERS%20BY%20PARTY%20AND%20PRECINCT.htm#STATEWIDE

[3] https://azsos.gov/sites/default/files/State_Voter_Registration_2020_General.pdf 

[4] https://azsos.gov/elections/results-data/voter-registration-statistics 

[5] https://elections.cdn.sos.ca.gov/ror/15day-gen-2020/county.pdf 

[6] https://www.sos.state.co.us/pubs/elections/VoterRegNumbers/2020/December/VotersByPartyStatus.pdf 

[7] https://portal.ct.gov/-/media/SOTS/ElectionServices/Registration_and_Enrollment_Stats/2020-Voter-Registration-Statistics.pdf 

[8] https://elections.delaware.gov/reports/e70r2601pty_20201101.shtml 

[9] https://dos.myflorida.com/elections/data-statistics/voter-registration-statistics/voter-registration-reports/voter-registration-by-party-affiliation/ 

[10] https://sos.idaho.gov/elections-division/voter-registration-totals/ 

[11] https://sos.iowa.gov/elections/pdf/VRStatsArchive/2020/CoNov20.pdf 

[12] https://sos.ks.gov/elections/22elec/2022-11-01-Voter-Registration-Numbers-by-County.pdf 

[13] https://elect.ky.gov/Resources/Pages/Registration-Statistics.aspx 

[14] https://www.sos.la.gov/ElectionsAndVoting/Pages/RegistrationStatisticsStatewide.aspx 

[15] https://electionstatistics.sos.la.gov/Data/Registration_Statistics/statewide/2020_1101_sta_comb.pdf 

[16] https://www.maine.gov/sos/cec/elec/data/data-pdf/r-e-active1120.pdf 

[17] https://elections.maryland.gov/pdf/vrar/2020_11.pdf 

[18] https://www.sec.state.ma.us/divisions/elections/download/registration/enrollment_count_20201024.pdf 

[19] https://sos.nebraska.gov/sites/sos.nebraska.gov/files/doc/elections/vrstats/2020vr/Statewide-November-2020.pdf 

[20] https://www.nvsos.gov/sos/elections/voters/2020-statistics 

[21] https://www.sos.nh.gov/sites/g/files/ehbemt561/files/documents/2020%20GE%20Election%20Tallies/2020-ge-names-on-checklist.pdf 

[22] https://www.state.nj.us/state/elections/assets/pdf/svrs-reports/2020/2020-11-voter-registration-by-county.pdf 

[23] https://klvg4oyd4j.execute-api.us-west-2.amazonaws.com/prod/PublicFiles/ee3072ab0d43456cb15a51f7d82c77a2/aa948e4c-2887-4e39-96b1-f6ac4c8ff8bd/Statewide%2011-30-2020.pdf 

[24] https://www.elections.ny.gov/EnrollmentCounty.html 

[25] https://vt.ncsbe.gov/RegStat/ 

[26] https://vt.ncsbe.gov/RegStat/Results/?date=11%2F14%2F2020 

[27] https://oklahoma.gov/content/dam/ok/en/elections/voter-registration-statistics/2020-vr-statistics/vrstatsbycounty-11012020.pdf 

[28] https://sos.oregon.gov/elections/Documents/registration/2020-september.pdf 

[29] https://www.dos.pa.gov/VotingElections/OtherServicesEvents/VotingElectionStatistics/Documents/2020%20Election%20VR%20Stats%20%20FINAL%20REVIEWED.pdf 

[30] https://datahub.sos.ri.gov/RegisteredVoter.aspx 

[31] https://sdsos.gov/elections-voting/upcoming-elections/voter-registration-totals/voter-registration-comparison-table.aspx 

[32] https://vote.utah.gov/current-voter-registration-statistics/ 

[33] https://sos.wv.gov/elections/Documents/VoterRegistrationTotals/2020/Feb2020.pdf 

[34] https://sos.wyo.gov/Elections/Docs/VRStats/2020VR_stats.pdf 

What’s next for The Lindahl Letter? 

• Week 135: Polling aggregation
• Week 136: Econometric models
• Week 137: Time-series analysis
• Week 138: Prediction markets
• Week 139: Machine learning election models

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

Thank you for tuning in to this audio only podcast presentation. This is week 133 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Automated survey methods.”

I’m still spending some time digging into notebooks. This time around the topic as you might have guessed for that inquiry is figuring out how to automate a survey or more pointedly some sentiment analysis. People are building automated phone surveys with interactive voice response (IVR) systems. The next wave of this technology will be hard to tell if it is a person or a bot. Seriously, those systems are going to keep getting better at a rapid pace. The new wave of generative large language models are going to make outbound call surveys better and probably more plentiful. When the outbound call survey plugins roll in for ChatGPT, Bard, and anybody can build one for Llama 2 if they are willing to serve up a custom model. At the same time, I’m entirely sure (and hopeful) that people will be using more advanced technology to block those phone calls as well.  

All right, let’s shift away from considering phone calls and start to dig around into some of the automated sentiment analysis techniques that exist. We are starting to see frameworks where you can ask these new series of ChatGPT type services to act as an agent for you and complete some type of tasking. One of the things that would be interesting to ask that type of agent to complete would be to evaluate sentiment about something. I’m sure brands would like to have some automated brand evaluation methods. This will inevitably be used for politics as well. Right now we are not to the point where everybody has plugins that allow agency for ChatGPT or other toolings at the moment. That really is coming very soon as far as I can tell. Between lower energy costs and the solid platforms being built, those changes together may enable the compute for this type of interaction to happen in very conversational ways with a computer in the next 5 years.

Right now you could start by messing around with Google Colab and use the forms options they have [1]. Completing some really solid sentiment analysis may require more than just focusing on the  Google Colab environment. You may want to go out to somewhere like Hugging Face to get some information on how to do this with some python code [2]. A nice place to go along this journey to get some sentiment analysis done would be to venture out to the world of Kaggle and access one of their notebooks for sentiment analysis [3].Another notebook that I liked was from notebook dot community and it shares some of the natural language processing basic of sentiment analysis in really good chunks that make it easy to understand the mechanics of how things are happening [4]. At this point in the process you are probably ready to start to do some work on your own to complete some sentiment analysis and I found the right Google Colab notebook for you to start work on designing your own sentiment analysis tool [5].

I was talking to somebody recently about the future of AI. My explanation may have not been what they expected to hear. Within the next couple of years I expect to see a lot of companies spin up and a lot of different creativity happening in the space. All of that will end up settling out into a commodified built in series of advancements. A lot of new features for applications and tooling will spin out off the great wave of AI builds that are happening now, but it will end up feeling more commonplace and build into technologies that exist now. These technologies will mostly supplement or augment things as we move forward. You will have to know how to interact with and work with the generative models that exist, but they are going to be built into the platforms and systems that end up winning out within the business world in the next couple of years.

Footnotes:

[1] https://colab.research.google.com/notebooks/forms.ipynb

[2] https://huggingface.co/blog/sentiment-analysis-python 

[3] https://www.kaggle.com/code/omarhassan1406/notebook-for-sentiment-analysis

[4] https://notebook.community/n-kostadinov/sentiment-analysis/SentimentAnalysis

[5] https://colab.research.google.com/github/littlecolumns/ds4j-notebooks/blob/master/investigating-sentiment-analysis/notebooks/Designing%20your%20own%20sentiment%20analysis%20tool.ipynb

What’s next for The Lindahl Letter? 

• Week 134: The chalk model for predicting elections
• Week 135: Polling aggregation
• Week 136: Econometric models
• Week 137: Time-series analysis
• Week 138: Prediction markets

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.