Sense, Nonsense, and Not-Sense

As requested by azurelunatic, who has people interested in how I can use my voice commands to make sure that if I need to go do something else, I don't miss out on anything that's happened on the TV or the movie in the interim.

This is a thing that requires four components to work in sequence - the voice assistant, the smart home brain, a programmable IR blaster, and an IR receiver attached to the device that's driving the video stream. So our workflow goes: Rhasspy → Home Assistant → Broadlink IR blaster → FLIRC receiver.

Rhasspy has featured from the very beginning of this series, and the intent scripts involved in getting voice input paired up with actions. The key insight into making this work was getting Rhasspy to understand numeric input and pass those parameters to Home Assistant, such that I can develop a sentence to train Rhasspy on that will collect the correct parameters, and an intent script and supplemental script on the Home Assistant side to format those parameters in a way that Home Assistant is expecting to run a timer with, running the appropriately-named timer with those parameters, and then repeating back what function it heard me invoke and any parameters that were passed to it. The time-out script accepts parameters, or, if none are provided, uses the default value that I've set for it (two minutes, which is the approximate length of a commercial break in the United States.) The time-out timer runs in the same way as any other timer helper or arbitrary timer, and there is an automation listening in Home Assistant for when that timer runs out that will run appropriate actions for the expiry of that timer.

What happens when that timer runs out is, essentially, the IR blaster fires a specific code to the FLIRC receiver interprets as having had the "pause" key pressed on a remote. Which sounds like it's easy, but it's not, because that IR blaster has no database of codes to look up to know what to send, and instead has to be taught what various IR codes are by having them beamed at it from a convenient remote. The FLIRC is just an IR receiver and interpreter. It can't transmit anything to the IR blaster to teach it anything. And while we have wireless keyboards for the convenience of the ten-foot interface, those keyboards don't transmit IR signals to a receiver. (nor do we want them to.)

What I do have, and have had, are programmable remotes, like a Logitech Harmony (no longer manufactured) or the Skip1s remote from the FLIRC folks. They do have databases of IR codes that can be downloaded into their remote, and therefore, that gives our IR blaster something to learn from. (Digression: The FLIRC can be used as a receiver to learn codes from an OEM remote and then teach those codes to the Skip1s, if the device isn't in the Skip database, but that's very much advanced fooling-about with both of those devices, and if you have the OEM remote, you can just teach the Broadcom device directly, rather than going through a Skip or other universal remote. I use programmable universal remotes because I want one remote to control all the things, rather than having to deal with multiple remotes to adjust things. And because while using the voice assistant is great, I don't want to have to use it (or a smartphone app) to do all the sound and picture-related remote control stuff.)

Actually getting IR codes into the IR blaster is actually an adventure in building another callable script that sets the remote into learning mode, and then goes through a sequence of "waiting for command" inputs, where in the script, I tell it what device it's learning how to control, and what the name of the command its learning is. Thankfully, the process on how to get a remote to learn, and then how to get it to send commands, is very well-documented in Home Assistant, far better than the underlying Python library that the integration is based on. It's a very manual process, but when set up right, I only have to do it once, and the IR blaster remembers what it's been taught and can then turn around and transmit the same thing. That's remote → IR blaster → FLIRC receiver, and then I can use both the remote or the IR blaster as needed.

Since Home Assistant can use the IR blaster to transmit to the receiver, the actual automation listening for the end of the time-out timer has one command associated with it - use the IR blaster to transmit a press of the "pause" key to the receiver. So long as the receiver is within range to receive, it will receive the keypress and do the associated action. Thus, the whole chain completes, voice command to script to timer to automation to IR transmission to keypress. It doesn't feel like a complex operation, because when it works, it works, and it doesn't invite contemplation of all the parts that have to work with each other to achieve the equivalent of pressing a key on a remote at the right time.

There are drawbacks to this setup. The biggest and most obvious one is that the pause key will only work on whatever item currently has focus on the receiving computer. If you tend to use picture-in-picture to watch multiple streams and/or listen to multiple sounds, the time-out keypress will only hit the one that the underlying operating system believes is currently in focus. As far as I know, there's no "pause all/resume all" that can be transmitted and interpreted from the things I have available. (After all, even though computers are very good at doing multiple tasks in sequence, humans are thought to be the kind of beings that only want to concentrate on one thing at a time, so why would you need something that pauses and/or resumes everything at once?) If that does actually exist, then I'll do my best to figure out how to incorporate it into scripting and see if I can teach it properly to the IR blaster / receiver that this is what I want.

Second, what the operating system believes is in focus and what the human believes is in focus are not always the same, so sometimes when I'm trying to get one thing to pause, it turns out the operating system has focus on something else. That requires manual intervention to get the focus where it should be, and at that point, you're already on the machine that needs to be stopped, so you may as well just click the right pause button yourself.

And third, not all programs, sites, streams, and the like respond correctly to a pause key, so there are occasions where I could push all the pause keys I wanted to, in whatever way I wanted to, and nothing would happen, because the site or program doesn't recognize it or has locked out that particular input from going through until some other thing is cleared, acknowledged, or otherwise managed. Thankfully, the number of situations where this has happened to me is pretty small, and there are sometimes some efficient and effective workarounds to this problem like the aforementioned picture-in-picture pop-out, which is usually pretty properly responsive when it's the thing that has focus, regardless of what it going on with the underlying website and its playback controls.

This idea is also very scalable and configurable - so long as you can get the remote to learn the appropriate command from another remote, or you have the appropriate base64 encodings that will work for the remote available so that you can drop it directly into the learned codes file, and the receiver on the other end knows what to do with the codes that it receives, you can basically scale this idea to anything that you might want to do with a remote control. I'd suggest using it only for things where you can either have a gap for execution in between commands, or if you have a workflow that can immediately begin recording a new input from the voice assistant after the last command has finished executing. Your own set-up will likely depend on what applications you want to control and their potential quirks, but it is rather nice to have that option in place where you can either let the commercials play and then pause before the action resumes, or give yourself until a breaking point and then have the media pause automatically so that you can task switch without FOMO or so that you can get your brain in gear to do the actual switching instead of just continuing to binge whatever it is that you're doing. (Kickstarting executive function with computers can be really helpful, if for no other reason than that they will do exactly what you tell them to do, on time.)

The Home Assistant is humming along smoothly at this point, probably because I haven't actually tried to do a whole lot with it in terms of extending functionality, increasing it, or writing new functions for it to achieve. It's behaving stably, for the moment, even as we keep everything updated and fix issues when they come up, like when the robot vacuum changed its IP address by one from the DHCP assignment and it took me weeks to figure out what the problem was. So this edition of Adventures in Computer Stuff is a lot about fixing and errors and using librarian skills to find solutions to those errors.

( Adventures in Error Correction and Workarounds )

So, once again, we have managed to make computers work and do what we want them to, mostly through the skills of search and persistence and finding a workaround when the direct method doesn't work. Not because of superior technical prowess or any of the skills where I would be able to directly understand what went wrong and know what commands to run immediately to confirm the issue or fix it. It's why having machines that you can play around with is vital to your learning, because then you have the freedom to try solutions and restore from backups if those solutions make a bigger mess than they had before. And you learn a little bit more about how the systems work every time that you succeed, and sometimes more every time that you fail. Have fun, everyone!

At the moment, most of the automations and controls that have been put in place for the Home Assistant are stable. There's a fair amount more of things to teach to the remotes so they can do more control matters in the rooms they're in, but for the most part, there hasn't been a whole lot of need to add more functions or figure out a method to do a thing that has been requested of me. (Which just means that a new thing hasn't caught our attention at this point and I haven't been tasked with figuring out how to make it work.)

What that does mean, however, is making sure that the things that are in place, the duct tape and string holding it all together, is still strong and properly reinforced. Sometimes that means making sure the glue script for turning a TV on and off over the MQTT/CEC bridge still works and has been put properly into place when the attached SBC changes to a different operating system. (And finding out, delightfully, that a different method of controlling the screen works just fine even if one of the libraries is tied up in the bridge.) Sometimes it means shuffling some methods over to make sure that there's still good remote access for when you want to do work on one of the other computers without disturbing the person who is in the room that's there. (Because the library you had been relying on to do that upgraded itself and the program that worked with it didn't upgrade to the new version.)

For this particular situation, though, it was the announcement from one of the API providers that I've been using for weather and other data saying "We're shutting down the always-free, needs no payment methods on file API access and moving all of you over to the still free for a limited amount of calls, but after that we charge you access instead." At which point I said, "Well, that's a service that I'm no longer going to be using." What that means, though, is that the one service that I was pulling weather, Air Quality Index, and Ultraviolet Index data from needed replacements. I was about to find out how good I had gotten, and how good my design of automations was, because I had to do some wholesale data source replacements.

( And the replacement is on )

Having found new sources, then I just had to rewire all the appropriate automations and commands to use the new data sources, which didn't take long, just repeatedly having to change from one source to another in all the places where it had been. And within the last two days, all of the automations that were running on new sources fired correctly, did their actions, and informed us about what was going on in the world and what might be a good action to take for us. So it's good to know that the things I have set up were set up correctly so that I could plug new and different sources into them and still have them work correctly. It gives me confidence both in the way the Home Assistant is set up and in my own abilities to get it to do what I want that this kind of source change was the work of a day to find, add, and rewire everything to run on different sources. This seems like another one of those situations where it looked easy to me and that may have involved more skill than I think it did.

The last time we checked in on the automation efforts, I'd done some cron job work to keep my Pi-Hole updated when there was a new version, having it check and run once a week for new Pi-hole system software. This time around, we're back in Home Assistant, trying to get some useful information ingested for new functionality, and to refine one of the sensors to be a little more accurate.

( Complex automation conditions, using API data, trend data, and JSON pathing )

( Fine-tuning the Internet Outage Sensor )

[VICTORY FANFARE.]

I'd say that these things are becoming more common for me, but that's not quite true. It's because each of these new projects is building on something I've already done before that they seem to be easier to achieve. When I'm building the new sensors and the URIs to poke the endpoints with from the documentation provided and the code I've already put into place, it's copy-paste-tweak, and the new knowledge is in the tweaking. Or practicing various skills and thinking through the processes that I want to happen means the tweaking generally works once I've picked up the component that's new for this enterprise. It might be because I'm working with a project that has both documentation and a robust forum culture, so I can look at how someone else has achieved their results and then tweak accordingly for my own purposes.

If I really wanted to test myself and what I know and possibly some other things like whether I can learn or remember how to program in a language and then generate an application as well as play around with APIs, Space Traders is right there to use as a deep dive to build a client for the game. I have other things to do than build a client for a game just because I'm feeling hubristic about how I can work with APIs and parsing JSON.

After adopting a "wait and see" approach toward whether Elon's Folly would go through with the threat to cut off all free API access (and smiling a knowledgeable smile when the threat was walked back to some degree,) and determining, as best I can tell, the status lights are still working properly, I had another idea of something I could get my external brain to do for me instead of requiring me to both remember and be motivated to do it. I have a book of excerpts and quotes that I have been meaning to use more as meditative prompts, and I added looking in it at a random page as a tickybox in my daily spread. I wasn't getting consistent results, based on a host of factors, most importantly whether I had time to do the thing as part of my morning routine or before going to bed. So, rather than relying on my internal brain to do the habit thing, I thought it might be possible to make the external brain do it. If the external brain could pull a random quote from a quotes file, and then sling the selected piece of wisdom to my tablet at some point during the day, I'd be more likely to examine it and meditate on it.

( And here we go, once again, into the rabbit hole. )

This is the end of this particular adventure in automation, getting an external brain to remind me of things to do, say, or contemplate throughout the day. It's probably not something in the specification or original conception of Home Assistant as a thing that can monitor IoT devices and then issue commands to them based on the data inputs and rules that I set up. Because Home Assistant is flexible, modular, and extensible, though, when I get wild ideas like this in my head, it turns out that there are pathways to implementation that I can find, code snippets to examine and reuse and modify, and as I try to make it work the way I want to, I learn a little bit more about the systems, their limitations, how to format things, how to construct queries, troubleshooting errors and issues, and more. It probably also helps that I'm going at it with the understanding that I can conceive of what I want in my head, that I have been successfully tinkering with machines and their configurations since the MS-DOS era, and the real difficulty of the task is finding enough material so I can create or modify code in such a way to make it work.

It works. There are things I want to do to improve it, but it works, and that's what's important.

( ETA: Postscript Victories )

Or: There Are No Timers Remaining

So, last we left this situation, I had managed to get a single arbitrary timer working and had a list of suggested improvements to make the arbitrary timer function more smoothly and effectively. There were three suggestions, and it turns out they're all related to each other.

( Here's the rest of the story )

All that just to be able to tell the thing to set a timer and have it behave appropriately. Still, I'm glad it's fine, and I learned a fair few things about the system and how to talk to it correctly in the process.

Oh, boy, did I have a fun time this weekend and vacation period getting things set up and trying to make my home automation more powerful and better able to respond to requests from the household. Two stellar examples emerge of problem-solving, scratching my head at the documentation, and trying things iteratively until something worked the way I intended it to. I feel like this is something I should be taking more pride in, since they were both successful, but mostly what's come out of them has been feelings of embarrassment at it taking this long, or that the solution that I've hit upon is probably Ugly and Wrong, or disbelief that I managed to make it work at all, since they look so long and so much effort to do. So, I'm laying these examples out for you, partly as a bid for your opinions on the matter so that I can recalibrate reality against brainweasels, and partly because I want to document that these things really did end up working.

( Example 1: The TV and the Pi )

( Example 2: The Sensor Screensaver Bash-Together (in Python) )

And those are two examples of Adventures in Home Automation (and its related functions), both of which required skills, problem-solving, consultation of the documentation, and iterative design until they worked correctly. If nothing else, I hope they were entertaining stories.