Recently I've been looking at a way to quickly build and deploy some wireless sensors to feed data into my home automation system and took a second look at the ESP8266 modules sitting in my parts drawer - these devices seem to be the quickest, cheapest and easiest way to get a remote sensor network up and running.

I have had some ESP-01 modules for a while now but haven't had a lot of opportunity to play with them (I've been working on other projects) and it seems that there have been a lot of improvements in the ecosystem since I last looked - a wider range of modules and development environments, far more projects and sample code for them and a vibrant community making use them in a range of applications.

If you haven't come across them before the ESP8266 is a small SOC (System On Chip) manufactured by Espressif. They feature a 32 bit processor, integrated 802.11 b/g/n WiFi and integrated SRAM. For a hobbiest the most common way to use them is in a module format, these modules add flash memory (from 512K to 4M) and either an onboard PCB antenna or a uFL connector for an external antenna.

ESP-01 and ESP-12 Modules

There is a wide range of modules available from several different manufacturers. Two of the most common ones, the ESP-01 and the ESP-12, are remarkably small and extremely cheap (an ESP-01 can be bought on eBay for around $AU 1.50 each).

The ESP-01 Module

The ESP-01 is the entry level version of the module: it features 512K of flash, an onboard antenna and exposes 4 GPIO pins.

ESP-01 Pinout

Pins are exposed on an 8 pin DIL header (which is unfortunately not breadboard friendly) and most of the GPIO pins have multiple functions so you need to take some care when designing your circuit.

The GPIO0 and GPIO2 pins must be pulled high when the device is powered up for normal operation, if GPIO0 is pulled low the device will enter bootloader mode to allow new firmware to be loaded on to the device.

The Tx (GPIO1) and Rx (GPIO3) pins are configured for serial communication on boot (and you will always get some debug information on the Tx pin on startup) but can be reconfigured when your program starts.

This page has a good overview of how to get the most out of the available IO pins. There is enough IO for simple remote sensors though if you make use of one wire devices like the DHT11 or DS18S20. And of course you could use I2C based devices to expand your IO capabilities.

The ESP-12 Module

If you do need more GPIO the ESP-12 may be the solution. This module has an onboard antenna, 4Mb of flash memory and 22 pins (including 11 GPIO, serial, analog input and a SPI interface).

ESP-12 Pinout

As with the ESP-01 the GPIO0 and GPIO2 pins are used to control boot loader entry so care must be taken when using them as IO pins, with 9 other GPIO pins available though this is not as big an issue.

The ESP-12 is even less breadboard friendly than the ESP-01, pins are exposed through 1.27mm pitch castellated edge connectors so you will definitely need a breakout board of some sort. Thankfully they are not that difficult to mill or etch and there are plenty available on eBay as well.

Software Development

The next step of course is to get your own code on the module. As I mentioned the ESP8266 has a very active community so that is the best place to look at to get the full range of options. In this post I will just cover some of the more common programming environments available.

ESP8266 SDK

Espressif themselves offer an official SDK and latest releases are announced in this forum. They seem to have a range of SDKs - one that includes an RTOS and runs user code as a task, another that is based purely on callback functions to interface with the WiFi stack and the 'AT Firmware' that allows an ESP8266 to be controlled like a modem. This is probably one of the more complex environments to set up and maintain but would give you the highest level of control over the chip.

There is also an alternative, open source (mostly) SDK available on GitHub which might be a little easier to install and setup.

Arduino Core

There is an ESP8266 Arduino Core available that can be installed with the Arduino Board Manager and allows you to use the Arduino IDE and tools for development.

The core provides a range of ESP8266 specific libraries and allows the use of a number of existing Arduino libraries as well (although you might have to make some modifications to them to get them to work). Some other interesting features it provides include using a portion of the flash memory as a filesystem and support for OTA (Over The Air) firmware updates.

If you are already familiar with the Arduino environment this is probably a good choice to get started with.

NodeMCU

This is pretty interesting, the NodeMCU firmware provides a Lua based interactive environment for developing ESP8266 software on the ESP8266 itself.

Lua is a dynamic scripting language with similar features to Javascript (but with a different syntax). Although not as well known as Javascript Lua has been very popular as a game scripting engine (most notably in World of Warcraft) and, as a result, there are a lot of resources available on the net to help you learn the language.

The development environment provided by NodeMCU is very similar to Node.js and provides the same event driven program model. The sample code below (from the NodeMCU site) shows how to implement a HTTP server.

-- a simple http server
srv=net.createServer(net.TCP)  
srv:listen(80,function(conn)  
    conn:on("receive",function(conn,payload) 
    print(payload) 
    conn:send("<h1> Hello, NodeMCU.</h1>")
    end) 
end)  

Apart from the software needed to flash the initial NodeMCU firmware the only development tool you will need is a serial terminal. If you have a web development background or are already familiar with Node.js this is probably a good place to start.

The Devil in the Details

Getting up and running with the ESP8266 is relatively easy and deploying one or two remote sensors won't be too difficult. However, if you want to deploy a lot of sensors and make it easy to maintain and duplicate your designs there are a few things that need to be taken care of.

Power Requirements

The ESP8266 is driven by a 3.3V supply and is relatively power hungry - according to Espressif the chip itself will draw 170mA while transmitting and the additional circuitry on the modules will add to this (my ESP-01 boards seem to peak at around 200mA).

This exceeds the capabilities of the 3.3V line on typical FTDI serial adapters so you won't be able to drive the module directly from them without running into problems. You will need a separate 3.3V regulator driven by the 5V USB power line and capable of providing 250mA to avoid module resets during operation.

The chip does support low power modes but they can be a bit tricky to manage especially if you want to maintain the wireless connection (the modem needs to receive and respond to the access point beacon messages so you can't turn it off completely). Even with careful coding and minimal data transfer you are probably looking at 20mA average current draw (15mA in sleep, 170mA when transmitting) - about 3 or 4 days of continous operation with a 3000mAh Li-Ion battery.

My intent is to use the devices for indoor sensors and notifications and power them using a USB charger which will avoid the power issues. If you are looking for something that can be powered by battery for long periods of time you should probably consider using a different technology.

Breakout Boards

As I mentioned, neither module is particularly breadboard friendly so building up prototypes can be problematic (and messy). A breakout board helps a lot and, as well as providing suitable pinouts for breadboard use, should include the basic support components as well - pull up resistors on the bootloader entry pins, a regulator and possibly pushbuttons for RESET and PROGRAM operations.

ESP-01 Breakout

I milled up a simple prototype for the ESP-01 and cleaned up the PCB layout for fabrication once I knew it was working. Those have been sent off to Seeed Fusion and with any luck 10 boards will be arriving soon.

My plan is to use the same PCB for prototyping a new sensor design as well as including it in the final deployed product - the custom parts of the sensor will be put on a daughter board and the two plugged together for deployment. These boards break out all connections to a 2.54mm pitch pin header, have the 3.3V regulator and pull up resistors on board and include a USB socket for power.

Initial Wireless Configuration

Setting the wireless configuration in the module is another issue that needs to be addressed. The simplest way of course is to simply hardcode the SSID and password in the firmware but this is probably not the best way to go about it. Apart from security issues (you accidently pushed your WiFi password to GitHub? Thanks!) if you change your network configuration at any time you will need to reflash all of your sensors with the new information.

The ESP8266 SDK supports something they call SmartConfig which allows you to use an Android App to set the SSID and password on the device - some sample code on using this feature using the ESP8266 Arduino Core is available here. Reports indicate that this feature is not particularly reliable though (and it's possibly not a portable solution if you start using a different wireless module).

Another alternative is to build a custom solution - the ESP can operate as an access point so if it has no wireless configuration (or the current configuration is failing for some reason) it can switch to AP mode and wait for someone to connect and configure the WiFi settings. This project on GitHub provides that functionality as an Arduino library.

I am still investigating what is available in this area - my ultimate goal is to be able to use an app on my phone or tablet to detect, connect to and configure wireless sensors. Ideally the system should be portable so it can be used with other WiFi enabled devices (CC3000 based modules or a Raspberry Pi Zero with a WiFi dongle) as well.

Sending and Receiving Data

The easiest way to communicate with a ESP8266 based module is to establish a TCP connection and treat it like a serial port - sending down data as text strings periodically. This would require a custom server at the other end that can accept the connections and understand the data format - something that could become messy and unmaintainable as the number of different types of sensors and data formats grows.

A more portable solution would be to use HTTP to transfer data (all of the development environments I mentioned above have some support for making HTTP requests) - just send the data as the body of a HTTP POST and have a web server handle it and dispatch the data to the appropriate place. Unfortunately this still requires custom code on the server side.

My plan is to use JSON as the data format and send it out over MQTT. The Mosquitto MQTT server is available for Windows, Linux and OS/X and existing tools like Node-RED or my SensHub server can be used as a rules engine to process the data and trigger events based on it.

There are still some details to be worked out - processing JSON in C++ is not as simple as it is in other languages and I would like to make use of mDNS-SD to find the address of the MQTT server to connect to rather than hardcoding it.

Next Steps

At this stage I am waiting for the breakout boards to arrive and prototyping some simple sensors to test out various ideas I have. I would like to have a simple template sketch for the Arduino core that provides a set of common features for any sensor:

  • WiFi configuration via a mobile phone application.
  • Sending and receiving data in JSON format over MQTT.
  • Service announcement (sensors on the network can be discovered by an application on a PC or phone).
  • Service discovery (sensors can determine the MQTT server to communicate with automatically).
  • Support for Factory Reset to clear existing settings.

Obviously there is still a fair bit of work to do but because the hardware design requirements are fairly minimal (most of the work involved is software related) I should be able to get some simple wireless sensors up and running quickly, quicker than finishing off the SensNode hardware and libraries anyway.

I'll post updates to my design and anything interesting I come across along the way - at the very least it should help those of you looking at implementing something similar.