Boot, connect to wifi, server requests in under 1 second with ESP-01

Making a hardware button that connects to wifi and sends a request off to a server is possible using Arduino. Light switches are (almost) immediate – but simple wifi switches easily take 8-10 seconds to connect & switch. How do you get that time down? Here’s my approach.

  1. Measure end-to-end - what’s the default time? Is it really that bad, or just sometimes bad?
  2. Split into parts
  3. Measure the parts
  4. Determine which parts should be optimized, and try options
  5. Combine the best options
  6. Debug why it doesn’t work and try again

The computer we’re using to run on is an ESP8266, model ESP-01. It’s tiny, cheap (< 1 USD), quite limited, and weirdly popular. Wifi buttons are popular projects, so I’m sure all of this isn’t new (it’s just new to me, so I was happy when this all worked out). I use MQTT since it’s one of the common setups for home automation. All measurements were done 10 times to get a sense of how stable they are.

Default code

The default code to connect & switch looks roughly like this (simplified):

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

WiFiClient wifi_client;
PubSubClient mqtt_client(wifi_client);

void setup() {
    WiFi.mode(WIFI_STA);
    WiFi.begin(WIFI_SSID, WIFI_AUTH);
    while (WiFi.status() != WL_CONNECTED) { delay(100); }
    mqtt_client.setServer(MQTT_SERVER, 1883);
    if (mqtt_client.connect(MQTT_CLIENT_ID, MQTT_USER, MQTT_AUTH)) {
        mqtt_client.publish(MQTT_TOPIC_ID, "hello");
    }
}

void loop() { }

The general flow is that it connects to wifi using the wifi’s name (WIFI_SSID) and password (WIFI_AUTH), waits for the connection to be stable, then connects to the MQTT server (MQTT_SERVER), authenticates with MQTT_USER & MQTT_AUTH, then sends a message to a “topic” on the server. It’s possible to measure the time in the code by checking the millisecond timer before & after. What you don’t see here is the boot time though.

Boot timing

To measure the boot time, I switch one of the output pins as the first step in the code, then I can measure how long it takes from power-on to when the pin is switched with an oscilloscope.

#include <Arduino.h>

void setup() {
  pinMode(PIN_TEST, OUTPUT); 
  digitalWrite(PIN_TEST, HIGH); 
}

void loop() { }

The measurement setup is something like this:

I set the trigger on the power connection (so it would start counting there) and manually measured where the change on the output pin happens.

Timings:

  • Boot time: 124.3ms (SD 1.9ms) - similar timings with different boards, low variance
  • Wifi + MQTT time: 5312ms (SD 3194ms) - very variable, fast times around 1.7s, slow times around 9s

Waiting 9 seconds for a light to switch on is annoying. I don’t even wait 2 seconds for a page to load.

Optimizing

The slowness comes from several places:

  • Building a wifi connection takes a lot of back and forth (scan for access points, check which ones with the SSID are closest, authenticate, get an IP address, get network information)
  • Connecting to a server requires several steps too (DNS, connect, authenticate - even when using plaintext MQTT)

The optimizations I looked at tried to address the slowest steps, further optimizations are likely possible too. For wifi, you can connect to a specific base station and channel directly, and save the scanning & selection process. The downside is if you move around, you might end up with a suboptimal, or out of range, base station. Or, your access point might hop channels and you end up with one that doesn’t actually work.

Wifi optimization

The timing & options looked like this:

Just WIFI:

  • Connect to SSID, use DHCP: 7473ms (SD 105ms) - notably slower than the fastest end-to-end times, which suggests the device may be doing optimizations in the end-to-end tests already
  • Connect to SSID, use a static IP / DNS: 1316ms (SD 33ms) - this is kinda usable already; is DHCP so slow, or is just getting it all set up so slow?
  • Connect to BSSID, use DHCP: 201ms (SD 60ms) - blinks. this is nice.
  • Connect to BSSID, use a static IP / DNS: 128ms (SD 0.4ms) - omg.

130ms to connect to wifi is pretty amazing. It feels like I’m probably missing something critical, why doesn’t my laptop do that? I’ll take it.

Adding MQTT:

  • Wifi with SSID, DHCP + MQTT to hostname: 5312ms (SD 3194) - as mentioned initially
  • Wifi with SSID, static IP + MQTT to IP address: 1432ms (SD 217ms) - DHCP & DNS lookups take time, but it’s not like the servers are changing IP addresses
  • Wifi with BSSID, static IP + MQTT: fails :-/

It turns out, the wifi code doesn’t actually connect when you specify the BSSID, it just acts like it could. In short, it’s a fast way to not make a connection. Annoying. It’s fast though, maybe there’s a way to make it work.

The solution ended up acting like we’re reconnecting after connecting, which does the actual connection.

    // ...
    WiFi.config(IPAddress(data.ip_address),
        IPAddress(data.ip_gateway), IPAddress(data.ip_mask), 
        IPAddress(data.ip_dns1), IPAddress(data.ip_dns2));
    WiFi.begin(data.wifi_ssid, data.wifi_auth, 
        data.wifi_channel, data.wifi_bssid);
    WiFi.reconnect(); // actually connect
    uint32_t timeout = millis() + TIMEOUT; // await connection or time out
    while ((WiFi.status() != WL_CONNECTED) && (millis()<timeout)) { delay(5); }
    return (WiFi.status() == WL_CONNECTED);

MQTT optimization

For the MQTT server, using a static IP is a quick help. You can look up the DNS of the MQTT host and just cache that.

    IPAddress mqtt_ip;
    int err = WiFi.hostByName(MQTT_SERVER, mqtt_ip);
    if (err==0) { 
        // couldn't get IP, we're doomed
        // it'll probably work next time though
    }

Additionally, since MQTT reuses the WiFiClient object for the connection, we can pre-connect to the server and potentially save some time there (unclear if this actually makes a difference though – perhaps not).

    // preconnect
    WiFiClient wclient;
    #define PRECONNECT_TIMEOUT 5000
    wclient.setTimeout(100);
    uint32_t timeout = millis() + PRECONNECT_TIMEOUT;
    while ((!wclient.connect(ip, port)) && (millis()<timeout)) { delay(50); }

    // mqtt stuff
    PubSubClient mqtt_client(wclient);
    mqtt_client.setServer(data.mqtt_host_ip, data.mqtt_host_port);
    if (mqtt_client.connect(MQTT_CLIENT_ID, data.mqtt_user, data.mqtt_auth)) {
        mqtt_client.publish(topic, value);
        // other topics
    }

Results

The total timing with this setup are pretty reasonable for a switch (well, for me), faster than this page loads.

  • Time to boot: 124.3ms (SD 1.9ms)
  • Time from boot to connect & send: 474ms (SD 53ms)
  • Time to refresh settings (when connections fail, or on initial startup): 7408ms (SD 79ms)

Hardware

(To be written up in another post.) I use a latching power switch that uses GPIO3 to power down completely, powering the device with 2xAA batteries.

  • Push button (>120ms) to turn on, it connects, sends request, blinks briefly.
  • If power is held, it re-initializes the flash.
  • If it’s held even longer, it enables OTA updates.

The device sends a topic for the WLED lights, as well as a topic for Home Assistant to tell it that it’s active (which is logged). Apparently multiple MQTT requests in the same session require minimal overhead, so sending both is fine. To enable the Home Assistant logging, I use MQTT autodiscovery in the re-initialization flow. You could just as well ping a random webserver, post a tweet (caveat Twitter API hassles), open the garage door, or do something useful.

Next steps

  • Do more fine-grained timing analysis.
  • Check what’s actually sent over the network.
  • Research if there’s a way to cache the MQTT server connection / authentication (suspicion is that the authentication handshake takes a while)
  • Move to MQTT over TLS
  • Measure power usage

Also the obvious step of cleaning up the code and dropping it on Github :-).

Comments / questions

There's currently no commenting functionality here. If you'd like to comment, please use Twitter and @me there. Thanks!

Tweet about this - and/or - search for latest comments / top comments

Related pages