Posts tagged with "iot"
I worked together with a few fine folks from my team on a very fun hackathon project, and I want to tell you about it.
Here’s our team…
The hackathon was themed on some relatively new products - namely Cognitive Services and the Bot Framework. Additionally, some members of the team were looking for some opportunity to fine tune their Azure Functions skills, so we went looking for an idea that included them all.
I’ve been mulling around the idea of using some of these technologies to implement an escape room, which as you may know are very popular nowadays. If you haven’t played an escape room, perhaps you want to find one nearby and give it a try. An escape room is essentially a physical room that you and a few friends enter and are tasked with exiting in a set amount of time.
Exiting, however, is a puzzle.
Our escape room project is called Cabin Escape and the setting is an airplane cabin.
Players start the game standing in a dark room with a loud, annoying siren and a flashing light. The setting makes it obvious that the plane has just crash landed and the players’ job is to get out.
Players look around in haste, motivated by the siren, and discover a computer terminal. The terminal has some basic information on the screen that introduces itself as CAI (pronounced like kite without the t) - the Central Airplane Intelligence system.
A couple of queries to CAI about her capabilites reveal that the setting is in the future and that she is capable of understanding natural language. And as it turns out, the first task of silencing the alarm is simply a matter of asking CAI to silence the alarm.
What the players don’t know is that the ultimate goal is to discover that the door will not open until the passenger manifest is “validated,” and CAI will not validate the manifest until all passengers are in their assigned seats. The only problem is that passengers don’t know what their assigned seats are.
The task then becomes a matter of finding all of the hidden boarding passes that associate passengers with their seats. Once the last boarding pass is located and the last passenger takes his seat, cameras installed in the seat backs finish reporting to the system that the passenger manifest is validated and the exit door opens.
Architectures of old were almost invariably n-tiered. We software developers are all intimately familiar with that pattern. Times they are a changing! An n-tier monolithic architecture may accomplish your feat, but it won’t scale in a modern cloud very well.
The architecture for cabin escape uses a smattering of platform offerings. What does that mean? It means that we’re not going to ask Azure to give us one or more computers and then install our code on them. Instead, we’re going to compose our application out of a number of higher level services that are each indepedent of one another.
Let’s take a look at an overall architecture diagram.
In Azure, we’re using stateless and serverless Azure Functions for business logic. This is quite a paradigm shift from classic web services are often implemented as an API.
API’s map to nodes (servers) and whether you see it or not, when your application is running, you are effectively renting that node.
Functions, on the other hand do not conceptually map to servers. Obviously, they are still running on servers, but you don’t have to be in the business of declaring how Functions’ nodes scale up and down. They handle that implicitly. You also don’t pay for nodes when your function is not actually executing.
The difficult part in my opinion is the conceptual change where with a serverless architecture, your business logic is split out into multiple functions. At first it’s jarring. Eventually, though you start to understand why it’s advantagous.
If any given chunk of business logic ends up being triggered a lot for some reason and some other chunk doesn’t, then dividing those chunks of logic in different functions allows one to scale while the other doesn’t.
It also starts to feel good from an encapsulation stand point.
Besides Functions, our diagram contains a DocumentDB database for state, a bot using the Bot Framework, LUIS for natural language recognition, and some IoT devices installed in the plane - some of which use cameras.
We use Azure Functions to update and retrieve the state of the game. Our Azure Functions are stateless, however we keep the state of every game stored in DocumentDB. In our design, every Cabin Escape room has its own document in the state collection. For the purpose of this project, we have one escape room that has id ‘officialGameState’.
We started by creating a ‘GameState’ DocumentDB database, and then creating a ‘state’ collection. This is pretty straight forward to do in the Azure portal. Keep in mind you’ll need a DocumentDB account created before you create the database and collection.
After setting up the database, we wrote our Azure Functions. We have five functions used to update the game state, communicate with the interactive console (Central Airplane Intelligence - Cai for short), and control the various systems in the plane.Azure Functions can be triggered in various ways, ranging from timed triggers to blob triggers. Our trigger based functions were either HTTP or timer based. Along with triggers, Azure Function can accept various inputs and outputs configured as bindings. Below are the functions in our cabinescape function application.
- GamePulse: * Retrieves the state of the plane alarm, exit door, smoke, overhead bins and sends commands to a raspberry piece * Triggerd by a timer * Inputs from DocumentDB - Environment: * Updates the state of oxygen and pressure * Triggered by a timer * Inputs from DocumentDB * Outputs to DocumentDB - SeatPlayer: * Checks to see if every player is in their seat * Triggerd by HTTP request * Inputs from DocumentDB * Outputs to DocumentDB and HTTP response - StartGame: * Initializes the state of a new game * Triggered by HTTP request * Outputs to DocumentDB and HTTP response - State: * Update the state of the game * Triggered by HTTP request * Inputs from DocumentDB * Ouputs to DocumentDB
A limitation we encountered with timer based triggers is the inability to turn them on or off at will. Our timer based functions are on by default, and are triggerd based on an interval (defined with a cron expression).
In reality, game is not being played 24/7. Ideally, we want the timer based functions triggered on when the game starts, and continue on an time interval until the game end condition is met.
An escape room is really just a ton of digital flags all over the room that either inquire or assert the binary value of some feature.
- Is the lavatory door open (inquire)
- Turn the smoke machine on (assert)
- Is the HVAC switch in the cockpit on? (inquire)
- Turn the HVAC on (assert)
It’s quite simply a set of inputs and outputs, and their coordination is a logic problem.
All of these logic bits, however, have to exist in real life - what I like to call meat space, and that’s the job of the controller. It’s one thing to change a digital flag saying that the door should be open, but it’s quite another to actually open a door.
The contoller in our solution is a Raspberry Pi 3 with a Node.js that does two things: 1) it reads and writes theh logical values of the GPIO pins and 2) it dynamically creates an API for all of those flags so they can be manipulated from our solution in the cloud.
To scope this project to a 3-day hackathon, the various outputs are going to be represented with LEDs instead of real motors and stuff. It’s meat space, but just barely. It does give everyone a visual on what’s going on in the fictional airplane.
My role at Microsoft is that of an evangelist, and that implies that we speak broadly to audiences about technology. Recently, however, my team is working one-on-one with organizations on exciting new Microsoft technologies - making sure they’re exactly what developers need.
One of the projects I’ve been working on recently is called Regatta championed by Baiyin Zhou (@baiyinvc) from Boston.
This entire idea is super fun. Imagine exercising on a rowing machine in a studio in a group session. Now imagine a visual on the wall depicting your rowing machine and those of the other session participants as real boats. As you heave and ho, you see your little boat moving across the screen.
There are just so many awesome scenarios that are possible at this point.
The visual of the boats alone is encouragement for you to work harder to compete against others in the class. But that’s pretty obvious isn’t it. What about the less obvious scenarios? What about being able to add in a phantom boat that represents the group’s average from yesterday’s workout? Now you’re not competing against your fellow rowers - you’re competing with them. You have to beat that phantom boat!
As soon as things in the real world - like our efforts to stay in shape - are digitized and turned into data, we get to do things like capture our progress, integrate with other fitness tracker systems, or even do data analysis to determine some deep learning from it.
This is the type of concept where digital systems really have an opportunity to affect our lives for the better. I keep saying that technology is not so interesting unless it improves our lives. For being such a technophile, I’m actually quite the skeptic, because all too often technology just gets in the way.
The Regatta Project envisions a future with a whole lot of rowing machines deployed with smart devices (the Raspberry Pi Zero at this point) attached to them, and assuring good communication with all of these devices is a great job for Azure IoT Hub.
Go check out the details of the [Regatta](https://microsoft.github.io/techcasestudies/iot/2016/12/10/regatta.html) project and let me know what you think with a comment below.
I beat my head against a wall for a long time wondering why I wasn’t able to do basic GPIO on a Raspberry Pi using Node. Even after a fresh image and install, I was getting cryptic node error messages when I ran my basic blinky app.
Lucky for me (and perhaps you) I got to the bottom of it and am going to document it here for posterity. Let’s go.
The head beating happened at a hackathon I recently attended with some colleagues.
The task was simple - turn on an LED. It’s so simple that it’s become the “hello world” app of the IoT world. There’s zero reason in the world why this task should take more than 10 minutes. And yet I was stumped.
After a fresh image of Raspbian, an install of NVM, and then a subsequent installation of Node.js 6.2.2, I wrote a blink app using a variety of modules. I used
onoff, and finally
johnny-five and the
None of these strategies were successful. Ugh. Node worked fine, but any of the libraries that accessed the GPIO were failing.
I was getting an obscure error about an
undefined symbol: node_module_register. No amount of searching was bringing me any help until I found this GitHub issue where nodesocket (thanks, nodesocket!) mentioned that he had the same issue and it was caused by an NVM install of Node and an accidental, residual version of node still living in /usr/local/bin. In fact, that was exactly what was happening for me. It was a subtle issue. Running node -v returned my v6.2.2. Running which node returned my NVM version. But somewhere in the build process of the GPIO modules, the old version (v0.10) of node from the /usr/local/bin folder was being used.
There are two resolutions to this problem. You can kill the old version of node by deleting the linked file using
sudo rm /usr/local/bin/node and then create a new one pointing to your NVM node. I decided, however, to deactive NVM…
…and then follow these instructions (from here) to install a single version node…
I like using NVM on my dev machine, but it’s logical and simpler to use a single, static version of Node on the pi itself.
EDIT (2016-12-14): Since writing this, I discovered the awesomeness of nvs. Check it out for yourself.
And that did it. I had blinky working in under 3 minutes and considering I get quite obsessive about unresolved issues like this, I had a massive weight lifted.
BTW, through this process I also learned about how the GPIO works at the lowest level on the pi, and I blogged about that at codefoster.com/pi-basicgpio.
I’ve been doing a lot of hacking on the Raspberry Pi, and I’ve written a few articles on the topic. I’ve assembled all of my posts here for easy access.
- Accidental Old Version of Node on the Raspberry Pi - I beat my head against a wall for a long time wondering why I wasn’t able to do basic GPIO on a Raspberry Pi using Node. Even after a fresh image and install, I was getting cryptic node error messages when I ran my basic blinky app.Lucky for me (and perhaps you) I got to the bottom of it and am going to document it here for posterity.
- The Most Basic Way to Access GPIO on a Raspberry Pi - I’m always looking for the lowest level understanding, because I hate not knowing how things work. On a Raspberry Pi, I’m able to write a Node app that changes GPIO, but how does that work? Turns out it’s pretty interesting. I’ll show you.
- Easy and Offline Connection to your Raspberry Pi - Getting a Raspberry Pi online is really easy if you have an HDMI monitor, keyboard, and mouse, but what about if you want to get connected to your Pi while you’re, say, flying on a plane?
- Wifi on the Command Line on a Raspberry Pi - How to configure wireless connections on your Raspberry Pi from the command line.
Getting a Raspberry Pi online is really easy if you have an HDMI monitor, keyboard, and mouse.
Subsequently getting an SSH connection to your pi is easy if you have a home router with internet access that you’re both (your PC and your pi) connected to.
But let’s say you’re on an airplane and you pull your Raspberry Pi out of its box and you want to get set up. We call that provisioning. How would you do that?
I’ll propose my method.
First, you need to plug your pi into your PC using an ethernet cable. If you’re a technologist of old like I am, you may be rummaging through your stash for a crossover cable at this point. It turns out that’s not necessary though. I was pretty interested to discover that modern networking hardware has auto-detection that is able to determine that you have a network adapter plugged directly into another network adapter and crosses it over for you. This means I only have to carry one ethernet cable in my go bag. Nice.
If you put a new OS image on your pi and boot it up, it already detects and supports the ethernet NIC, so it should get connected and get an IP automatically.
Here comes the seemingly difficult part. How do you determine what the IP address of your pi is if you don’t have a screen?
The great thing is that the pi will tell you if you know how to listen.
The means by which you listen is called mDNS. mDNS (Multicast DNS) resolves host names to IP addresses within small networks that do not have a local name server. You may also hear mDNS called zero configuration and Apple implemented it and felt compelled (as they tend to) to rename it - they call it Bonjour.
This service is included by default on the Raspberry Pi’s base build of Raspbian, and what it means is that out of the box, the pi is broadcasting its IP address.
To access it, however, you also need mDNS installed on your system. The easiest way I am aware of to do this is to download and install Apple’s Bonjour Print Services for Windows. I’m not certain, but I believe if you have a Mac this service is already there.
Once you have mDNS capability, you simply…
The name raspberrypi is there because that’s the default hostname of a Raspberry Pi. I like to change the hostname of my devices so I can distinguish one from another, but out of the box, your pi will be called raspberrypi. The
.local is there because that’s the way mDNS works. And finally, the -4 is an argument that specifically requests the IPv4 address.
If everything works as expected you’ll see something like…
Again, my pi has been renamed to
cfpi1, but yours should be called
raspberrypi if it’s new.
My system uses 192.168.1.X addresses for my wireless adapter and 169.254.X.X for my ethernet adapter.
So that’s the information I needed. I can now SSH to the device using…
I could just use
ssh firstname.lastname@example.org to remote to it, but I’ve found that continuing to force this local name resolution comes with a little time cost, so it’s sometimes significantly faster to hit the IP address directly. I only use the mDNS to discover the IP and then I use the IP after that.
Provisioning a Raspberry Pi usually includes a number of system configuration steps too. You need to connect it to wireless, set the locale and keyboard language, and maybe turn on services like the camera. If you’re used to doing this through the Raspbian Configuration in XWindows, fear not. You can also do this from the command line using…
Most everything you need is in there.
You may also be wanting to tell your pi about your wifi router so it’s able to connect to via wireless the next time you boot up. For that, check out my post at codefoster.com/pi-wifi. Actually, if you’re playing a lot with the Raspberry Pi, you might want to visit codefoster.com/pi and see all of the posts I’ve written on the device.
I hate hooking a monitor up to my Raspberry Pi. It feels wrong. It feels like I should be able to do everything from the command line, and the fact is I can.
If you’re pulling your Raspberry Pi out of the box and are interested in bootstrapping without a monitor, check out my other post on Easy and Offline Connection to your Raspberry Pi.
Afterward, you may want to set up your wifi access - that is, you want to tell your pi about the wireless access points at your home, your coffee shop, or whatever.
Doing that from the command line is pretty easy, so this will be short.
You’re going to be using a utility on Raspbian called
wpa_cli. This handles wireless configuration and writes its configuration into
/etc/wpa_supplicant/wpa_supplicant.conf. You could even just edit that file directly, but now we’re talking crazy talk. Actually, I do that sometimes, but whatever.
…to see what the current status is. If you get
Failed to connect to non-global ctrl_ifname: (null) error: No such file or directory, that’s just a ridiculously cryptic error message that means you don’t have a wifi dongle. Why they couldn’t just say “you don’t have a wifi dongle” I don’t know, but whatever.
If you do have a wifi dongle, you’ll instead see something like…
Yay! You have a wireless adapter, which means you likely have a wifi dongle plugged into a USB port. It says here that the current state is
INACTIVE. That’s because you’re not connected to any access points.
To do so, you need to run scan, but at this point, you may want to enter the wpa_cli interactive mode. That means that you don’t have to keep prefixing your commands with wpa_cli, but can instead just type the commands. To enter interactive mode, just do…
To get out at any time just type
Now do a scan using…
It’s funny, because it appears that nothing happened, but it did. Use…
…to see what it found.
This scanning step is not necessary, by the way, there’s a good chance you already know the name (SSID) of your access point, and in that case you don’t need to do this.
Next you create a new network using…
You’ll get an integer in return. If it’s your first network, you’ll get a 0. That’s the ID of the new network you just created, and you’ll use it on these subsequent commands.
To configure your network do this…
Something I read online said that as soon as you enter this, it would start connecting, but I had to also do this to get it to connect…
Now there’s one more thing. If you’re like me, you don’t just connect to a single AP. I connect from home, my mifi, my local coffee shop, from work, etc. I want my pi to be able to connect from any and all of those networks.
Adding more networks is as easy as following the instructions above multiple times, but you want to set one more network property - the priority. The priority property takes an integer value and higher numbers are higher priority. That means that if I have network1 (priority 1) and network2 (priority 2), and when my pi boots it sees both of those networks, it’s going to choose to connect to network2 first because it has the higher priority.
Okay, that does it.
If you want to see everything I’ve written about the Raspberry Pi, check out codefoster.com/pi
I’ve been hacking on the Raspberry Pi of late and wanted to share out some of the more interesting learnings.
I think people that love technology love understanding how things work. When I was a kid I took apart the family phone because I was compelled to see what was inside that made it tick. My brother didn’t care. If it made phone calls, he was fine with it. I had to understand.
Likewise, I knew that I could use a Node library and change the GPIO pin levels on my Raspberry Pi, but I wanted to understand how that worked.
In case you’re not familiar, GPIO stands for General Purpose Input/Output and is the feature of modern IoT boards that allows us to controls things like lights and read data from sensors. It’s a bank of pins that you can raise high (usually to something like 3.3V) or low (0V) to cause some electronic behavior to occur.
On an Intel Edison (another awesome IoT board), the platform developers decided to provide a C library with mappings to Node and Python. On the default Edison image, they provided a global node module that a developer could include in his project to access pins. The module, by the way, is called libmraa.
On a Raspberry Pi, it works differently. Instead of a code library, a Pi running Raspbian uses the Linux file system.
When you’re sitting at the terminal of your pi (either hooked up to a monitor and keyboard or ssh’ed in), try…
You’ll be taken to the base of the file system that they chose to give us for accessing GPIO.
The first thing to note is that this area is restricted to the root user. Bummer? Not quite. There’s a way around it.
The system has a function called exporting and unexporting. Yes, I know that unexport is not a real word, but alas I’m not the one that made this stuff up, and besides, who said Linux commands had to make sense?
To access a pin, you have to first export that pin. To later disallow access to that pin, you unexport it.
I had a hard time finding good documentation on this, but then I stumbled upon this znix.com page that describes it quite well. By the way, this page references “the kernel documentation,” but when I hit that link here’s what I get…
Now keep in mind that to follow these instructions you have to be root. You cannot simply sudo these commands. There is an alternative called gpio-admin that I’ll talk about in a second. If you want to just become root to do it this way, you do…
If you get an error when you do that, you may need to first set a password for root using
To export then, you do this…
And the pin number is the pin name - not the header number. So pin GPIO4 is on pin 7 on an RP2, and to export this you use the number 4.
When you do that, a virtual directory is created inside of
gpio4, and that directory contains virtual files such as
active_low. These files don’t act like normal files, by the way. When you change the text inside one of these files, it actually does something - like perhaps the voltage level on a GPIO pin changes. Likewise, if a hardware sensor causes the voltage level on a pin to change, the content of one of these virtual files is going to change. So this becomes the means by which we communicate in both directions with our GPIO pins.
The easiest way, then, to read the value of the
/sys/class/gpio/gpio4/value file is…
To write to the same file, you have to first make sure that it’s an
out pin. That is, you have to make sure the pin is configured as an output pin. To do that, you change the virtual
directionfile. Like this…
That’s a fancy (and quick) way to edit the contents of the file to have a new value of “out”. You could just use
nanoto edit the file, but using
echo and the direction operator (
>) is quicker.
Once you have configured your pin as an output, you can change the value using…
Now that I’ve described the setting of the direction and the value, you should know that there’s a shortcut for doing both of those in one motion…
There’s more you can do including edge control and logic inversion, but I’m going to keep this post simple and let you read about that on the znix.com page.
Now, although it’s fun and satisfying to understand how this is implemented, and it might be fun to manipulate the pins using this method, you’ll most likely want to use a language library to control your pins in your app. Just know that the Python and Node libraries that change value are actually just wrappers around these file system calls.
When you call the
write() method in this library, it’s just calling the file system.
So there you have it. I hope you feel a little smarter.
I’m going to keep much of my latest project under wraps for now, but I need your help designing one piece of it.
I’m looking to create a device that is small and light and will dispense one piece of candy at a time programmatically.
Here are the requirements…
- Light - no more than 1oz
- One at a time - it needs to dispense a piece of candy one at a time. If I have to constrain it to hold only round candy of a set size, then so be it, but ideally it would hold candy of varying size and shape. Ideally it will hold Runts bananas.
- It has to operate on a 3.7V power supply
- It has to be an electro-mechanical assembly that I can control with an IoT device (an easy req to fulfill I think)
- It should hold some reasonable number of candy pieces - say at least 10.
So far I’ve thought of the following…
- An Archimedes screw that “pumps” candy up from the bottom of a hopper (like this)
- A wheel that sits horizontal and on each incremental rotation lets one candy piece at a time fall into a hole in the top and lets another fall out a hole at the bottom
- A screw that sits on a horizontal axis that is loaded with candy pieces (one per screw rotation… not in a hopper) and pushes one piece out at a time when rotated 360 degrees. A bit like snack vending machines.
If you can come up with any other brilliant ideas, I’d be glad to hear them.
Thanks in advance.
I hack in Microsoft’s Maker Garage whenever I can. The Maker Garage is a relatively new addition to The Garage, which you may already be familiar with. The Maker Garage is a space where Microsoft Employees can come build things - hardware things, electronic things, software things, any things.
I’m a recent convert to the hardware side. I studied a bit of both hardware and software in my undergraduate Computer Engineering course many years ago, but it’s been almost entirely software for me since then. This return to resistors, capacitors and NPN transistors is a lot of fun, and in my role as a Developer Evangelist at Microsoft, it’s excellent for engaging with beginners, young folks, and anyone looking for a bit of a change from the sometimes humdrum enterprise app engineering space (hand raised!).
My last project in the Maker Garage was not a typical Azure-backed, IoT project, however. This time the customer was my 3 year old son. My wife notified me that we needed some random little figurines to complete a birthday gift, and why should we mail order figurines from across the globe when we can print them?! That’s right… no good reason.
There are a number of 3D printers in the Maker Garage, but the most recent adoption is a Form 1+ lithographic printer by FormLabs.
I first learned about the Form 1 watching the Netflix special Print the Legend some weeks ago. If you have a Netflix subscription, I recommend the documentary.
The Form 1 does not use the more popular filament extrusion technique employed by most of the 3D printers you see in the headlines. Instead, it uses stereolithography. Instead of melting plastic filament and shooting it out of a nozzle, a stereolithographic printer builds your model in a pool of liquid resin. The resin is photo-reactive and cures when exposed to a UV laser. This means you can shoot a tiny spot in the resin and it will harden. That’s perfect for turning goo into a ‘thing’.
The ‘things’ I decided to print were sea creatures. My son likes them, and they’re easy to find on Thingiverse.
I found an octopus, a submarine (not a creature I understand), a crab, a ray, an angler fish, a seahorse, and a whale.
Here’s the model of the octopus opened in 3D Builder that comes with Windows - a very slick touch-smart 3D model viewer, editor, and printer driver…
You could intuit even if you didn’t know that 3D models, like 2D vector graphics, can be scale to most any size, but unlike 2D vector graphics, they don’t always maintain their fidelity. 2D vector graphics tend to fully describe shapes. A circle is described as a circle that renders fine when you set its diameter to a millimeter or a kilometer.
3D models in .stl format - the common format for 3D printers - on the other hand are meshes. These meshes can lose their fidelity when you scale your model up. In my case here, though, I’m scaling these guys way down, so quality is not a problem.
I use Autodesk Fusion 360 for my modeling, but in this case I was able to import the .stl files directly into the software FormLabs ships with the printer. It’s the best printer driver software I’ve seen yet from a manufacturer and allowed me to scale, orient, layout, and print all seven of my characters with no trouble at all. I wish I had a screen shot of the layout just before I hit print, but I didn’t save that.
I scaled each character to somewhere between .75” and 1” on its longest axis.
You have a chance to choose the resolution of your print, and your decision has a linear impact on the print time. I chose .1mm layers at an estimated 1 hour print instead of upgrading to .05mm and adding another hour.
And here are my results.
I was impressed.
Take a look at a closeup of the whale…
You can see the layering. That’s quite fine, and as I mentioned, it could have been double that had I had more patience.
The piece that really impressed me was the tiny seahorse…
That’s just an itty bitty piece and it’s looking so smooth and detailed. Check out the small snout, dorsel fin, and tail. The same is true with the octopus legs.
One of my favorite things about the output of a lithographic printer is the transparent material. I much prefer a transparent, color-agnostic piece to the often random filament color that you end up with in community 3D printers. You can get tinted resin for the lithographic printers too, but the results are still models that are at least translucent.
I hope you enjoy this report out. Feel free to add suggestions or questions in the comments section below.
Alright, this is going to be fun. The process is going to be fun, and the end game is going to be even more so.
Let me paint a picture of the final product. I have a monkey toy on the table before me. I then hold my Microsoft Band up to my mouth and talk into it like Maxwell Smart. I say two simple words - “Monkey, dance!”
And in no time flat, the monkey toy obeys my command and is set into motion.
The whole thing reminds you of Tweet Monkey and it should. This is Tweet Monkey’s older and slightly more involved brother.
Where Tweet Monkey was a device to cloud scenario, Command Monkey is a Cortana to phone app to cloud to device scenario. Where Tweet Monkey relied on the Twitter Streaming API (which is very cool), Command Monkey involves our very own streaming API using web sockets.
I know it sounds like it’s going to be a lot of code, but it’s really not. You’ll see. Let me just say too that if for some reason you don’t have any interest in going through this step by step, then don’t. Just go grab the code on GitHub, because that’s how we roll.
I’m going to be using the free community version of Visual Studio and the Node.js Tools for Visual Studio. You could obviously use anything beyond an abacus to generate ASCII, so let’s not get opinionated here. Use what you love. It should go without saying that you’ll need Node.js installed to make this work. That can be found at nodejs.org.
I’m going to host my Node.js project in Azure and in fact, I’m going to get it there in a rather cool way. I’m going to use the cross platform CLI for Azure to create the site and then I’ll use git deployment to publish the app.
The architecture diagram is going to look something like this…
The part about speaking the command into a Microsoft Band just looks like special sauce, but you get that for free with a Band. The Band already knows how to talk to Cortana on your phone.
You will need to teach Cortana to talk to you app, so let’s do that first. Let’s build a Windows Phone app.
CommandMonkey.phone. Call the solution just
CommandMonkey. Like this…
In order to customize Cortana, we need to define a voice command and that requires the Microphone capability. To add that, double click on your package.appxmanifest, go to the Capabilities tab, and check Microphone…
Now we need to create a Voice Command Definition (VCD) file. This file defines to Cortana how to handle the launching of our app when the user talks to her. Here’s what you should use…
The interesting bits are lines 6 and 9-14. The combination of the prefix with the command name means that we’ll be able to say “Monkey dance” and Cortana will understand that she should invoke the Command Monkey app and use “dance” for the command.
You’ll notice that I have a second command in the phrase list for the command - chatter. I don’t currently have a mechanical monkey capable of both dancing and chattering, but you can imagine a device capable of doing more than one action and so the capability is already stubbed out.
Next we need to register this VCD file in our phone app. When we do this and then install the app on a phone, Cortana will then have awareness of the voice capabilities of this app. She’ll even show it to the user when they ask Cortana “What can I say?” To do that, open the
js/default.js file and add this to the beginning of the
And there’s one more step to completing the Cortana integration. We need to actually do something when our application is invoked using this voice command.
Still in the
default.js, add this after the
if statement that is looking for the
Now let’s talk about what that does.
if statement we hung that
else if off of is checking to see just how the app was launched. If the user clicked the tile on the start screen, then the value will be
activation.ActivationKind.launch. But if they activated it using Cortana then the value will be
activation.ActivationKind.voiceCommand, so this is simply how we handle that case.
The way we handle it is to access the parsed semantic using the event argument and then to send whatever command the user spoke to the CommandMonkey.azurewebsites.net website.
How did that website get created you might ask? I’m glad you did, because we’ll look at that next. As for the phone project, that’s it. We’re done. It’s not fancy, and in fact if you run it, you’ll see the default “Content goes here” on a black screen, but remember, we’re keeping this simple.
Visual Studio is able to hold multiple projects in a single solution. So far we have a single solution (called CommandMonkey) with a Windows Phone project (called CommandMonkey.phone).
Now we’re adding the web service.
Find the solution node in Visual Studio’s Solution Explorer and right click on it and choose Add | New Project…
Now install the express and socket.io Node modules. You can either do it the graphical way or the command line way.
The graphical way is to right click on npm in the .service project and choose Install New npm Packages… Then search for and install express and socket.io. For each leave the Add to package.json checked so they’ll be a part of your project’s package.json.
Now the command line way. Navigate to the root of this project at a command line or in PowerShell and enter
npm install express socket.io --save
Now open up the app.js and paste this in…
This is not a lot of code for what it’s doing. This is…
- creating a web server
- setting up web sockets using socket.io
- setting up a handler for when clients call the ‘setTarget’ event
- defining a route to /api/command which calls the “target” client with the specified command
Okay, the service is done, but it’s still local. We need to get this published.
We’re going to use Azure Websites feature called git deployment.
There’s something interesting about our project though. The git repository would exist at the solution level and include all of our code, but the folder that contains the Node.js project that we actually want published to Azure is in a subdirectory called CommandMonkey.service. So we need to create a “deployment file”. To do that just create a file at the root of the project called .deployment and use this as the contents…
And in typical fashion, we’re going to have a few files that we have no interest in checking in to source control, so make yourself a .gitignore file (again at the root of the CommandMonkey solution) and use this as the content…
Now git commit the project using…
Now you need to create an Azure website. Note, you need to already have your account configured via the Azure xplat-cli in order to do this. I’ll consider that task outside the scope of this article and trust you can find out how to do that with a little internet searching.
And of course, you can’t use CommandMonkey, because I’ve already used that one, but you can come up with your own name.
--git parameter, by the way, tells the create command to also set up git deployment on the remote website and will also create a git remote in the local repository so that you’ll be able to execute the next line.
To publish the website to azure, just use…
…and that will push all of your code to a repository in Azure and will then fire off a process to deploy the site for you out of the CommandMonkey.service directory which you specified earlier.
There’s one thing you need to do in the Azure portal to make this work. If anyone knows how to do this from the xplat-cli, be sure to drop me a comment below. I’d love to know. You need to turn on web sockets. Just go to your website in the portal, go to the configuration tab, and find the option to turn on web sockets. Easy.
Whew, done with that. Not so hard. Now it’s time to write the Node.js app that’s going to represent the device.
As is often the case, the device project is the simplest project in the solution. It does, after all, only have one job - listen for socket messages and then turn the relay pin high for a couple seconds. Let’s go.
In Visual Studio, right click on the solution and Add | New Project… and add another blank Node.js app. This time call it CommandMonkey.device…
Using the technique above for installing npm packages, install the
socket.io-client npm package.
You can see that this code is connecting to our web service in Azure -
It also requires and initializes the Cylon library so it can talk to the hardware in an easy, expressive, and modern way.
work method is like Cylon’s
ready method, and so that’s where we’ll invoke the
setTarget event. This socket event request for the server to save this socket as the “target” socket. That just means that when anyone triggers messages on the server, this is the device that’s going to pick them up. You may want to create this as an array and thus allow for multiple devices to be targets, but I’m keeping it simple here.
Finally, it handles the ‘command’ event that the server is going to pass it and simply raises the digital pin for 2 seconds.
To make this work, you need to deploy this project to your device. I use an Intel Edison, but this will work on any SoC (System on a Chip) that will run Linux. I am not going to repeat how to setup the Edison or deploy to it. You can find that in my series of Intel Edison posts indexed at codefoster.com/edison.
Once you get the code deployed to the device, you then run the CommandMonkey.phone project on your phone emulator or on your device. I run mine directly on my device so I can talk to my Microsoft Band.
And that should pretty much do it. If I haven’t made any gross errors in relaying this and you haven’t made any errors typing or pasting it in, then you’re scenario should work like mine. That is… you hold the action button on your Microsoft Band and say…
…and a very short time later you get a message with the command on the screen.
I hope you’re mind is awhir like mine is with all the ideas for things you could do with this. We have real time device to device communication going on, and users really get excited when they utter a command and a half a second later something is happening in front of them.