elections – Nels Lindahl — Functional Journal

Thank you for tuning in to this audio only podcast presentation. This is week 138 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Prediction markets & Time-series analysis.”

We have been going down the door of digging into considering elections for a few weeks now. You knew this topic was going to show up. People love prediction markets. They are really a pooled reflection of sentiment about the likelihood of something occuring. Right now the scuttlebut of the internet is about LK-99, a potential, maybe debunked, maybe possible room temperature superconductor that people are predicting whether or not it will be replicated before 2025 [1]. You can read the 22 page preprint about LK-99 on ArXiv [2]. My favorite article about why this would be a big deal if it lands was from Dylan Matthews over at Vox [3]. Being able to advance the transmission power of electrical lines alone would make this a breakthrough.

That brief example being set aside, now people can really dial into the betting markets for elections where right now are not getting nearly the same level of attention as LK-99 which is probably accurate in terms of general scale of possible impact. You can pretty quickly get to all posts that the team over at 538 have tagged for “betting markets” and that is an interesting thing to scroll through [4]. Beyond that look you could start to dig into an article from The New York Times talking about forecasting what will happen to prediction markets in the future [5].

You know it was only a matter of time before we moved from popular culture coverage to the depths of Google Scholar [6].

Snowberg, E., Wolfers, J., & Zitzewitz, E. (2007). Partisan impacts on the economy: evidence from prediction markets and close elections. The Quarterly Journal of Economics, 122(2), 807-829. https://www.nber.org/system/files/working_papers/w12073/w12073.pdf

Arrow, K. J., Forsythe, R., Gorham, M., Hahn, R., Hanson, R., Ledyard, J. O., … & Zitzewitz, E. (2008). The promise of prediction markets. Science, 320(5878), 877-878. https://users.nber.org/~jwolfers/policy/StatementonPredictionMarkets.pdf

Berg, J. E., Nelson, F. D., & Rietz, T. A. (2008). Prediction market accuracy in the long run. International Journal of Forecasting, 24(2), 285-300. https://www.biz.uiowa.edu/faculty/trietz/papers/long%20run%20accuracy.pdf

Wolfers, J., & Zitzewitz, E. (2004). Prediction markets. Journal of economic perspectives, 18(2), 107-126. https://pubs.aeaweb.org/doi/pdf/10.1257/0895330041371321

Yeah, you could tell by the title that a little bit of content related to time-series analysis was coming your way. The papers being tracked within Google Scholar related election time series analysis were not highly cited and to my extreme disappointment are not openly shared as PDF documents [7]. For those of you who are regular readers you know that I try really hard to only share links to open access documents and resources that anybody can consume along their lifelong learning journey. Sharing links to paywalls and articles inside a gated academic community is not really productive for general learning.

Footnotes:

[1] https://manifold.markets/QuantumObserver/will-the-lk99-room-temp-ambient-pre?r=RWxpZXplcll1ZGtvd3NreQ

[2] https://arxiv.org/ftp/arxiv/papers/2307/2307.12008.pdf

[3] https://www.vox.com/future-perfect/23816753/superconductor-room-temperature-lk99-quantum-fusion

[4] https://fivethirtyeight.com/tag/betting-markets/

[5] https://www.nytimes.com/2022/11/04/business/election-prediction-markets-midterms.html

[6] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+prediction+markets&btnG=

[7] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=election+time+series+analysis&oq=election+time+series+an

What’s next for The Lindahl Letter?

Week 139: Machine learning election models
Week 140: Proxy models for elections
Week 141: Election expert opinions
Week 142: Door-to-door canvassing

If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

Thank you for tuning in to this audio only podcast presentation. This is week 137 of The Lindahl Letter publication. A new edition arrives every Friday. This week the topic under consideration for The Lindahl Letter is, “Tracking political registrations.”

Trying to figure out how many republicans, democrats, and independents are registered in each state is actually really hard. It’s not a trivial task. Even with all our modern technology and the extreme power of the internet providing outsized connectedness between things and making content accessible to searches. Even GPT-4 from OpenAI with some decent plugins turned on will struggle to complete this task.Your best searches to get a full list by state are probably going to land you into the world of projections and surveys. One that will show up very quickly are some results from the Pew Research which contacted people (300 to 4,000 of them) from each state to find out more data about political affiliation [1]. They evaluated responses into three buckets with no lean, lean republication, or lean democrat. That allowed the results to evaluate based on sampling to get a feel for general political intentions. However, that type of intention based evaluation does not give you a sense of the number of voters within each state.

It opened the door to me considering if political registration is even a good indicator of election outcomes. Sports tournaments rarely play out based on the seeding. That is the element of it that makes it exciting and puts the sport into the tournament. To that end back during week 134 I shared the chalk model to help explore a hypothesis related to registration being predictive. At the moment, I’m more interested to see how proxy models for predicting sporting events are working. Getting actual data to track changes in political registrations is an interesting process. ChatGPT, Bard, and Bing Chat are capable of providing some numbers if you prompt them properly. The OpenAI model GPT-3.5 has some older data from September 2021 and will tell you registered voters by state [2]. I started with a basic prompt, “make a table of voter registration by state.” I had to add a few encouraging prompts at some points, but overall the models all 3 spit out results [3]. The Bing Chat model really tried to direct you back to the United States Census Bureau website [4].

This is an area where setting up some type of model with a bit of agency to go out to the relevant secretary of states websites for the 30 states that provide some data might be a way to go to build a decent dataset. That would probably be the only way to really track the official data coming out by state to show the changes in registration over time. Charting that change data might be interesting as a directional view of how voters view themselves in terms of voter registration in a longitudinal way. People who participate in Kaggle have run into challenges where election result prediction is actually a competition [5]. It’s interesting and thinking about what features are most impactful during election prediction is a big part of that competition. Other teams are using linear regression and classification models to help predict election winners as well [6]. I was reading a working paper from Ebanks, Katz, and King published in May 2023 that shared an in depth discussion about picking the right models and the problems of picking the wrong ones [7][8].

To close things out here I did end up reading this Center for Politics article from 2018 that was interesting as a look back at where things were [9]. Circling back to the main question this week, I spent some time working within the OpenAI ChatGPT with plugins trying to get GPT-4 to search out and voter registration by state. I have been wondering why with a little bit of agency one of these models could not do that type of searching. Right now the models are not set up with a framework that could complete this type of tasking.

Footnotes:

[1] https://www.pewresearch.org/religion/religious-landscape-study/compare/party-affiliation/by/state/

[2] https://chat.openai.com/share/8a6ea5e7-6e42-4743-bc23-9e8e7c4f79c5

[3] https://g.co/bard/share/96b6f8d02e8e

[4] https://www.census.gov/topics/public-sector/voting/data/tables.html

[5] https://towardsdatascience.com/feature-engineering-for-election-result-prediction-python-943589d89414

[6] https://medium.com/hamoye-blogs/u-s-presidential-election-prediction-using-machine-learning-88f93e7f6f2a

[7] https://news.harvard.edu/gazette/story/2023/03/researchers-come-up-with-a-better-way-to-forecast-election-results/

[8] https://gking.harvard.edu/files/gking/files/10k.pdf

[9] https://centerforpolitics.org/crystalball/articles/registering-by-party-where-the-democrats-and-republicans-are-ahead/

What’s next for The Lindahl Letter?