Once upon a time: Unleash
Let the hack commence
It is not all work and no play here at Zengenti (https://www.zengenti.com/). We love getting together for a BBQ, a walk or a casual game of cricket but one thing we really look forward to is the annual hackathon. If you are not familiar with a hackathon, don’t worry it’s nothing sinister and no hacking of banks goes on. Every year we stop monitoring the Kanban boards, cancel our stand up meetings and take two days to work on something (anything) we find interesting. Many great ideas have come from this annual ritual that have benefited both our customers and staff at Zengenti.
We are not crazy, so we still answer the phones and provide support. But the focus becomes a free for all, to play with things that aren’t part of our normal work with Contensis. This actually represents a large investment for the company to have the entire staff take two days to mess about with things that they find interesting or diverting, but historically this has paid dividends. This year five of us took a look at feature toggling.
You turn me on
What is feature toggling? Conceptually it is not that hard to grasp. When you are developing software you want to add features. You also want to add them as soon as you can, to see if they work. Building software is an exploratory process. It is not like building a house. Houses are made up of bits and pieces that are known to be reliable. All the things that are lying around at the builders merchant. I am not saying that building a house is easy, I have had a go and clearly it isn’t, but software development is always about doing something new.
Every piece of software is a prototype intended to solve a unique problem and one is never entirely sure if a new feature will solve part of that problem. The key point is that it is uncertain. The challenge is introducing unknown features to a working piece of software without messing up some other feature. The solution is to introduce each new feature with a built in on/off switch. The new feature can be lying dormant, then when you are ready, you can turn it on. And equally, if things don’t go to plan you can turn it off again. These on/off switches are colloquially known as feature flags because they allow you to toggle a feature on or off.
Unleash the flags
[Full disclosure: We convinced Unleash to send us some tee-shirts in exchange for writing about our experience. The other mantra at Zengenti is “Can we have a tee-shirt?” and we apply the same reasoning to our colleagues in other organisations. Though I will be honest there is a healthy trade in Zengenti socks going on as well. It seems that the price of a programmer at Zengent is a pair of socks, but that is another story.]
The host with the most
The key to beginning with Unleash is the hosted service, which is the home for your feature flags and provides a web interface to turn them on or off. There is no need to host the service yourself. The team at Unleash aren’t completely benevolent; they have to eat like the rest of us, so for a fee they will provide a hosted service for you. Allowing you to off load the effort and cost of hosting.
Monty the Python
Setting up the Unleash server is just a preamble to the main event of using feature flags in our software. Like many software developers we find Python and Flask are a handy means to create an API. We drive a lot of interrelated stuff via our API. It is an environment where deploying new changes can have knock on effects elsewhere in our systems.
Which is exactly the situation where feature flags would provide an extra layer of protection, giving us the confidence to deploy changes knowing we can roll them back in an instant. Previously we had used combination configuration files and rolling back deployments to earlier versions. It worked but late on a Friday night when you are tired and want to undo a change so you can attend to it on Monday morning, it required more effort than we would like. Our hope was that Unleash would reduce that effort. Fortunately it does.
Unleash provides an extension https://github.com/Unleash/Flask-Unleash that smooths the introduction of feature flags for Flask APIs. Initially we were confused by the
apparent complexity of the Unleash extension. It is only contacting the Unleash server for information about the feature flags, so why the complexity? Turns out that the Unleash extension is asynchronously contacting the Unleash server in the background. So when you request a flag, it has already been retrieved.
Why bother going to this effort? Look mum! No delays! This gives us the intriguing possibility of testing the performance of features as well as their functionality. It becomes a relatively trivial task to compare the performance of new features with old features. You can flip the switch and try it out with no reloads and no reboots. Any delays or lags can be attributed to the feature on, not to the feature flag.
Once again the people at Unleash have provided an SDK for our convenience and enjoyment https://github.com/Unleash/proxy-client-react. The eagle eyed among my readers will notice that this is a client for a proxy. So you also need a proxy to go with the client https://github.com/Unleash/unleash-proxy . At first glance, this seems like yet another layer of extra complexity that one would rather do without. But have faith, the people at Unleash have done the thinking for us. A Reactjs client typically runs in the browser with the client. So it is running in an environment where the user has access to all the settings and can view any calls to the Unleash server. And the calls to the Unleash server include your secret key. Well you wouldn’t want anyone taking a look at your feature flags, so you can’t go handing out your key to anyone with a browser. The solution is a proxy between the browser and the Unleash server. Like the Python SDK, the proxy server can be polling the Unleash server in the background, keeping an up to date version of your feature flags. So it can react quickly (pun intended), but it also keeps your secret key out of the grubby little hands of your users. This does sound a tad complicated as I write it but for practical purposes it wasn’t. We embedded the Unleash proxy into the Nodejs server, which was serving our Reactjs application and then the Reactjs proxy client did it’s work. Now we can turn features on and off in our Reactjs applications as well.
Teaming with flags
Normally our hackathons are just a festival of nerdiness. We like a bit of technology and the hackathons are an opportunity to indulge our desires. But using a feature flag service raises some interesting possibilities that weren’t available with our old approach using configuration files. Now that we are freed from the configuration files, two or more applications can use the same set of feature flags, irrespective of where they are or what languages they are written in. Feature flags can be a means to tie together teams as well as technologies. Now we can turn a feature on and off in a Reactjs application, that makes use of a Python API endpoint and is turned on and off with the same feature flag.
The Unleash server interface means that someone else (who isn’t one of the developers) can also do the switching. The developers don’t have to be around for the launch of a new feature, they can be getting on with something else. A manager or stakeholder that has closer contact with our customers can take over that responsibility. That isn’t to say the developers are out of touch with the process. We have set up our Unleash server to post changes to one of our Slack channels. Now whenever a feature is toggled on or off, the channel is notified. No one needs to go on to high alert but it just keeps everyone informed. Introducing feature toggling has had the side effect of improving and smoothing communications between our teams.
Let the toggling commence
We have been using Unleash since before Christmas, so not long really. It would be overstating it to say that it has saved our bacon. It has, however, made our lives much easier and more relaxed. Sure there is always a bit of tension with releasing a new feature, but knowing that it can be safely rolled back takes the pressure off. Which is exactly what it is supposed to do. We expect to be toggling happily ever after.
If you are a Zengenti customer and want to start making use of feature flags give us a call. We would be happy to share our experience and take you through the process we used. Otherwise contact the team at Unleash, who knows you might even be able to get a tee-shirt out of them too.