The Plan: an AR view of the IoT

My goal is to build an Augmented Reality view over a house full of IoT sensors and actuators.

The approach is to start with some Raspberry Pies running sensors or actuators and exposing them with REST interfaces, then to turn them into BLE beacons and use that to drive and position an Android Augmented Reality app.

The beacons give distance information to place the AR view, as well as broadcasting the URLs linking to the R-Pies' objects being viewed, to be fetched over WiFi.

Those objects will carry coordinate information to anchor them in the combined view; the 3D objects of many R-Pies will be linked up into a virtual place overlaying your house. Then you will be able to see what's going on in the R-Pies virtually in the form of 3D objects at those URLs.

You spark up the Android app and, as you wander around the house, you would see an AR view of your IoT world: you would see the state of all your sensors and actuators and could interact with them. 

You could see the temperature or occupancy in every room, check the doors and windows, read the weather centre in the garden, see if you have a phone message and check on the house electricity consumption. You could touch actuators to turn on the lights, set the temperature, set your home media centre to record a show, etc.

Then you would also be able to see other, completely virtual, objects such as 3D signs, whose text is some derivative of the state of the IoT world, such as "all doors and windows locked" or a big red button labelled "turn off all the lights".

You could switch to moving by on-screen controls, so that you could interact with your whole house from the comfort of your armchair or bed. By setting a bookmark, you could even visit your house when away, and wander around it from the poolside.

Ultimately, you'd have R-Pies all around your house, perhaps talking to devices directly from the I/O pins, perhaps through Bluetooth, ZigBee or Z-Wave. Each R-Pi would publish its objects into the shared, linked-up virtual world.

The interactions and mashups between your things and to and from the all-virtual objects will be user-programmable with simple rules that match conditions and set new states.