Crawling all (most) of the web's robots.txt comments

Starting from this tweet … View tweet … I hacked together a few-lined robots.txt comment parser. I thought it was fun enough to drop here. Crawling the web for all robots.txt file comments curl -s -N http://s3.amazonaws.com/alexa-static/top-1m.csv.zip \ >top1m.zip && unzip -p top1m.zip >top1m.csv while read li; do d=$(echo $li|cut -f2 -d,) curl -Ls -m10 $d/robots.txt | grep "^#" | sed "s/^/$d: /" | tee -a allrobots.txt done < top1m.

Crawling all (most) of the web's robots.txt comments »

Redirecting from Blogger with JavaScript

While it’s possible to redirect a whole site from Blogger to the exact same URLs, redirecting to similar URLs is hard. Let’s say you want to redirect: from: https://yoursite.blogspot.com/2012/01/blogpost-name.html to: https://newsite.com/blogpost-name The way to do this is to drop a snippet of JavaScript into your Blogger site’s template. Adding a redirect like this also makes it impossible for you to view your blog, so be careful :-). Also, adding a redirect like this means you won’t be able to edit your site’s theme using the simple tools (you’ll need to edit the HTML) - usually that’s no big deal since you want the old site to disappear, right?

Redirecting from Blogger with JavaScript »

Tiny (still-flakey) USB keyboard with ATMEGA 32u4

After making a USB keyboard with an ATTINY85 and noticing it wasn’t the “yellow of the egg”, I decided to try at making an ATMEGA 32u4 version. Overview What I was looking to make was (recap from previously): a simple USB keyboard with 1 key reprogrammable tiny mechanical keyboard key cheap enough to give away actually works debuggable The “cheap” aspect was mostly to justify making & buying some :).

Tiny (still-flakey) USB keyboard with ATMEGA 32u4 »

Coronavirus / Covid-19 notifications could not be activated: FAILED_SERVICE_DISABLED

Got a new phone and wanted to set up a local Coronavirus / Covid-19 exposure app but it just can’t be turned on? The solutions are simple: either wait a few hours for everything to update, or wait a few years, for developers to make usable error codes. Grr I installed SwissCovid on a new Android phone and tried to activate proximity tracing. It threw a useless error message with:

Coronavirus / Covid-19 notifications could not be activated: FAILED_SERVICE_DISABLED »

Wifi-control for a 5050 LED strip with ESP32

The 5050-type LED strips are pretty cheap, and can (sometimes) be controlled by ESP32’s fairly easily. The rough idea is to connect the individual color pins to a NPN transistor and to control them with an ESP32 PWM output pin. 5050 LED strips are always set to the same color, you can’t control them individually. There are different voltages of strips available - we’re using 5V since they seem to be pretty cheap, and they can just be powered by a USB cable.

Wifi-control for a 5050 LED strip with ESP32 »

Xiaomi Mijia thermometer & hygrometer in Home Assistant (LYWSD03MMC) on Hass.io

Connecting a thermometer & hygrometer (humiditymeter?) to Home Assistant can be a bit of a hassle. Who doesn’t want one in every room? There are lots of ways to get them set up, but it’s easy to make things nicely-complicated & expensive. Who doesn’t love spending more money than you need? I picked up a pack of Xiaomi Mijia sensors (or LYWSD03MMC, when you get to know them), they run on a coin-cell battery for about a year, have BTLE / Bluetooth Low Energy transmission of their status, and are pretty cheap.

Xiaomi Mijia thermometer & hygrometer in Home Assistant (LYWSD03MMC) on Hass.io »

Setting up MQTT with Auto-Discovery on Home Assistant (settings)

After you’ve installed things for MQTT in Home Assistant, it’s time to make some settings. 1. Enable debug logs. I don’t actually have much insight into what this does, but whatever. It sounds good. I did it. It didn’t hurt. In the configuration.yaml file (see previous post for information on how to edit files), add: logger: default: warning logs: homeassistant.components.mqtt: debug … and then reload the settings by restarting. Configuration -> Server Controls -> Check Configuration.

Setting up MQTT with Auto-Discovery on Home Assistant (settings) »

Setting up MQTT with Auto-Discovery on Home Assistant (software setup)

MQTT on Home Assistant is one of the most annoying things to figure out. The documentation is extremely minimal, their forum is filled with links to the documentation when people ask about not being able to understand the documentation. I’m keeping notes & warnings here. There are a few elements of MQTT that are important for this. You might already have some of this set up, if you followed some general Home Assistant getting started guides.

Setting up MQTT with Auto-Discovery on Home Assistant (software setup) »

Picking an ESP32 dev board

There are a bunch of ESP32 dev boards out there, and if you don’t want to make your own, you probably just want to pick one that just works well & is cheap. But which one is for you? Learn from my failures in picking boards. Updated December 2020 (well, that’s when this was written) For most use cases, these boards are equivalent and differ by pin count, board size, lipo support, USB port, and random accessories.

Picking an ESP32 dev board »

Tiny (flakey) USB keyboard with ATTINY85

It would be neat to have a dedicated key to mute myself in these ever-present video meetings. I don’t have a lot of clues when it comes to electronics, but this was an interesting first project with a PCB of my own. Overview What I was looking to make was: a simple USB keyboard with 1 key reprogrammable tiny mechanical keyboard key cheap enough to give away The last item was probably just a self-justification - if I make a board, I might as well make a bunch of them.

Tiny (flakey) USB keyboard with ATTINY85 »