Posts for March 2025

An in depth guide on getting started with Reinforcement Learning by OpenAI.  

I'm very late to the trend of AI playing Pokemon gameboy games, but I just started playing Pokemon Red myself on the iOS Delta emulator, and have been having lots of fun. To be clear, this is my first time doing anything with Pokemon. It wasn't something I was into as a child, but am for some reason discovering it as an adult and enjoying it.

I just wanted to make a quick post to show how I got the PokemonRedExperiments project running on my MacBook Pro M4 using uv.

Full steps:

# Clone the repo
git clone https://github.com/PWhiddy/PokemonRedExperiments.git

# Install ffmpeg
brew install ffmpeg

# Copy ROM to git root path
cd PokemonRedExperiments
cp /path/to/pokemon-red.gb PokemonRed.gb
# Validate rom is valid. Should produce ea9bcae617fdf159b045185467ae58b2e4a48b9a
shasum ./PokemonRed.gb

# Set up python environment
cd baselines
uv venv --python 3.10
uv pip install -r requirements.txt

# Start the pre-trained RL agent
uv run ./run_pretrained_interactive.py

I first tried using Python 3.12, as the README suggested using Python 3.10+, but I found that there are package dependency conflicts with 3.12, so I changed my uv command to use 3.10 and everything worked. This is why I love uv. I can very easily try out other versions of python and not worry about messing up other projects.

Something happened to me a couple of weeks ago that gave me pause. I found a bright, green leaf in the middle of my driveway. This might not sound like a notable event, but it's the end of winter, and all the trees around me are barren, awaiting spring to create their buds.

To me, this looks like a full, summertime leaf, maybe from a maple tree. It really disoriented me, I had to stop what I was doing and ponder what was going on. Is reality breaking? Is the simulation throwing exceptions? Is the increase in compute usage in our world (due to increases in usage of AI and crypto) causing our host reality to lag out and add sprites in the wrong places? These are the thoughts that flooded my brain.

I've been thinking about that leaf for a while. I finally realized I live in the 21st century and can identity the leaf with my phone, and maybe that would offer a clue to my mystery. And yeah, it's Common Ivy, an evergreen plant. Mystery solved.

While this was a silly, and to me, quite funny little freak out, it has me wondering what things other people have stumbled across in the world that seemed so completely out of place, that it could only be by some non-natural means. I'm sure this happens to every one at some point in their lives.

I just learned about a new open-source tool from Microsoft called MarkItDown. 

This seems similar to pandoc, but instead of any being able to take any formatted document type and convert it to any other type, it only outputs to markdown. It can be used as a standalone CLI tool or as a python library.

I'm particularly interested in converting HTML to markdown, so that I can take public documentation online and convert it into a markdown file, which can be more effectively consumed by LLMs. I was playing around with this idea last week during a hackathon, where I wanted to take the query language specification for WIQL that is online and turn it into a compact prompt, so the LLM can more reliably create WIQL queries for me.

To get the HTML for the web page, I use Simon Willison's tool shot-scraper to dump the HTML of the webpage, then pipe it into markitdown

shot-scraper html https://learn.microsoft.com/en-us/azure/devops/boards/queries/wiql-syntax | markitdown > wiql.md

This produces a file called wiql.md (link to gist with unmodified output). It's certainly not perfect, the first 300 lines (out of around 1000), are not related to the documentation, and is just extra HTML that isn't needed. This could probably be mitigated by passing an element selector to shot-scraper, so it doesn't dump the unrelated HTML of the page. But it's not hard to delete those lines manually, and then the final result is pretty good. It looks fairly similar to the original web page.

edit: Here is the one-liner to only dump the relevant part of the page.. You have to wrap the output of shot-scraper in a <html> so markitdown can infer the input type.

echo "<html>$(shot-scraper html https://learn.microsoft.com/en-us/azure/devops/boards/queries/wiql-syntax -s .content)</html>" | markitdown -o wiql.md

MarkItDown also supports plugins, so you can extend it to support other file formats. I've only played around with this a little bit, but I think it will be handy to have a quick and easy way to convert more documents to markdown. I'm particularly interested in the pdf and docx input types as well.

A friend sent me this paper regarding a security vulnerability in Chinas Great Firewall. The hack is quite interesting and worth reading about, but this quote is what stuck with me 

Spent my free time adding image support to the blog today. I'm using Azure Storage accounts, as it's what I know, and I did not feel like diving into S3 buckets right now, although I probably should at some point.

Once again, .NET Aspire proved its usefulness. I kicked off the changes by adding the Azure Storage hosting and client integrations via Aspire, then added a Minimal API POST endpoint so I had a place to upload the images to. GitHub Copilot is very useful in these kinds of tasks. I can write a comment block describing what I want the endpoint to do, all the edge cases I think it should handle, then hit tab and get 90% of what I wanted. I go through and tweak the rest, then I'm testing almost immediately.

The last piece of the puzzle was hooking up my new endpoint to the Trix editor. Fortunately, the Trix dev team provided a nice Javascript file to show how to hook up the event listeners and use an XHR object to post the image back to the server. Considering my lack of experience with Javascript, I was ecstatic to get this working in only a few iterations.

I edited my last post to add a picture from my bike ride, so I know it's working, but I better add one to this post too :)

I broke my wrist back in January when I fell skiing down a run in the early season. Snow conditions were pretty bad and the moguls on the run had really built up. The resort really hadn't been able to groom the runs yet. I did the dumbest move possible and on my first run of the day decided to go down a black diamond. Classic move on my part.

I did a turn to go around a mogul and didn't realize how big a drop off there was. I lost balance and fell down it onto hard packed snow. I broke the fall with my left hand and immediately knew something was wrong, but did not consider that I broke anything... I got back up and continued to ski down. I did fall one other time but it was a typical lose-your-balance type fall. The only issue was getting back up, when I realized I couldn't put any weight onto my left hand. That's when I knew I was cooked. I managed to ski the rest of the way down to the bottom and found the nearest ski patrol. They put my hand in a splint and that's when I saw the massive swelling. I knew my day, if not season, was over.

The last two months have been tough, I had been really looking forward to skiing and getting better this season. I also haven't been able to bike much. But I'm now close to fully recovered, the bone is healed and I got the all-clear to start biking again. Today was my first day back on the bike in two months, my biggest gap in between bike rides in years. What a day it was, a beautiful "fake spring" kind of day.

Here's to looking forward to many more rides soon.