Considering equations in ML papers

During the course of the 4th of July I got a chance to read a few PDFs of papers. Being able to write within the academic tone that papers usually have is a skill. Sometimes the papers include great scientifically based research, but are difficult to follow due to being poorly written. In the machine learning space this is one of those things that happens and could be compounded due to the mathematics the author is trying to share. Within the abstract and introduction things will start out on the right academic footing, but then as the mathematics start to get introduced things will veer off into the wild unknown. Most of the mathematics that gets shared within the machine learning space is not a provable theorem or something that you can easily break down and check. Every single time somebody starts to walk me through a series of equations in a paper I start to evaluate the work. Most of the time you cannot check the actual output given that the equations in the paper are implemented as code. Generally, that code is not shared so you cannot work your way backward from the code to the paper or the other way around to figure out what the author did for the actual delivery of the mathematics. 

The other piece of the puzzle that often worries me is that the equation presented in the paper is theoretical as an implementation and they are using an algorithm built into software that was already written. Within this example the author did not implement the mathematics within the code and probably is not deriving a perfect reflection of equation and implementation in the paper. Being able to run code as a part of a software package and being able to work through the equation using LaTeX or some other package to articulate it within an editor are very different things. I would have to work it out with pen and paper and then bring it over to the paper after the fact. Generally, I would be working with a known transform or algorithm within the machine learning space. It would be unlikely that I would be advancing the equations denoting the mathematical machinations beyond the original efforts. Within the coded implementation I might do something in an applied way that could ultimately be represented as a novel piece of mathematics. However, most researchers would be better off presenting the code vs. trying to transform the code back into a series of mathematical representations within an academic paper. 

It might very well be a good topic for a research paper to do an analysis of the equations presented in mathematical form in the top 100 machine learning papers by number of citations. It might be interesting to see the overlap in the equations across the paper. Alternatively it might be very illuminating to see if no overlap exists given that it’s possible outside of a shared statistical foundation the researches are not working from a shared mathematical base. A book on the mathematics of machine learning would be interesting to read for sure assuming it built from the basics to the more complex corners of the academy of knowledge surrounding the topic.

Working on manuscript updates

My writing schedule has been updated for Sunday on a go forward basis. The schedule now includes a morning writing session to review my Substack post and work on academic articles for 1-3 hours, publishing a note on LinkedIn about the last Substack post, and sending a Tweet about my last Substack post. Working within that writing schedule will probably help increase the readership of the newsletter. To pick up my subscribers the content is going to need to be posted to Twitter and to LinkedIn. This week I even elected to login to my highly dormant Facebook account to share the link. 

Over the last two months my writing productivity has really plummeted. That is something that needs to be rectified here one day at a time. Reversing a trend like that is just about spending the time at the keyboard to write larger blocks of prose. I have a manuscript in progress that is entitled, “On Machine Learning: The Lindahl Letters of 2021.” I’m going to publish my entire set of 2021 Substack posts in one long manuscript format. All of the content has been put into a basic manuscript draft and I’m currently working on starting at the beginning and reworking the content from start to finish before publication. This is one of those things that I’m probably going to need to send to my editor before final publication. Right now the manuscript is about 165 pages long, but that could easily get longer before publication. 

Earlier this morning I just could not manage to get back into the flow of writing. A trip to the donut store and a stop to get some coffee happened, but a bunch of time writing never occurred. Getting back on schedule with a writing routine has to be my primary focus moving forward. Yesterday I created a draft post with two solid paragraphs of content. For some reason that was where my productivity stopped. I’m about to watch the Kansas City Chiefs game on Sunday night football. During that game my plan is to try to rework the content and get it ready for publication on Friday.

Catching up on some writing

This week I sat down and completed the draft of the week 15 edition of The Lindahl Letter. That one is titled, “What are people really doing with machine learning?” The draft is now complete for publication on Friday May 7, 2021 at 17:00 hours Denver time. You may have guessed that today it seemed like a good idea for me to finish up that draft instead of starting my day with this ongoing writing project to produce a page of prose each and every morning. That is how my day started and it turned out well enough. I’m pretty sure the upcoming missives are going to be a little shorter for a bit until I have time to catch up and work ahead of my weekly delivery schedule. Right now I have a list of topics that pushes things out to week 37 and some of them are better than others. 

Yesterday or at some point recently, I lost a YouTube subscriber. It was probably my latest guitar video that displeased them and they left by taking the time to click unsubscribe. My general interests are technology and guitars. Certainly that is driven by my interest in the intersection of technology and modernity. Within that space I enjoy learning about civil society, the public square, artificial intelligence, machine learning, and all things technology related. Outside of that general space I tend to watch videos about guitar pedals and technology. While the two topics are slightly related most people may not share my interest in a mixture of such things. I’m not really worried at all about growing my audience. For the most part the videos that I create and the articles that I write are for my own purposes. They formally help me refine my thoughts and engage in the practice of doing things. Writing dense long misses of prose every day on a weblog is probably not the best method of communication to grow an audience. Some people have found success on Substack writing and sending out newsletters. That is probably directly related to finding a shared community of interest that wants to learn more and engage. Communities of place, circumstance, and interest develop naturally and the ones based on interest are probably the most passionate fan bases for a newsletter. 

The other day I was wondering what the actual audience size was for people who are interested in technical track applied machine learning content. My estimate of that audience size is about 10,000 people worldwide that might be actually interested in that topic. At any given time for any given publication a much smaller portion of that base would actually engage and consume that content. We have reached a point in the process of learning about machine learning where the creation of related content outpaces the ability of the community to consume it. That creates islands of specialization and all of the challenges and oddities that are produced from microcosms.