Case Study: Doctors Without Borders(Médecins Sans Frontières) & NixOS
Médecins Sans Frontières (MSF), also known as Doctors without Borders, was faced with a problem: They had dozens of servers deployed throughout the planet, often in makeshift clinics in some of the harshest, most remote places. They needed a way to manage these servers and keep them updated with the latest versions of their software. The IT team at MSF Belgium discovered a hidden gem: NixOS, which is a free and open source Linux distribution ideal for such management. Let’s look at how MSF tackled this challenge.
About MSF
MSF employs over 45,000 people working across the globe, running on funds from over 7 million individual, private donors[1]. To date they have provided medical care to over 16 million patients, and over 1.3 million patients have been admitted to their clinics and hospitals[2]. Operating in remote regions of the planet that lack infrastructure, they still manage to bring modern healthcare to patients in need.
The Challenge
MSF’s IT team at the Brussels Operational Centre (OCB) maintains roughly 100 servers deployed to field hospitals, remote medical centers and the cloud. These servers are rugged NUCs (which stands for Next Unit of Computing) that operate with minimal connectivity.
In the past, managing these servers was difficult because each one had to be configured and updated manually, which was costly, time consuming, and error-prone, especially in low-bandwidth areas with limited connectivity.
The IT team searched for different solutions and decided NixOS was the most promising.
What is NixOS?
NixOS is a Linux distribution built around the Nix package manager. Instead of relying on scripts, NixOS uses what’s called a “declarative” approach to configuration files which basically describe how the system should look, while letting NixOS handle the details of how to get there.
By adopting NixOS, MSF was able to ensure reproducibility, meaning the different servers would all be able to run exactly the same, and avoid “configuration drift” as NixOS can verify the machine is running with the intended configuration. Further, if anything goes wrong during an update, NixOS can easily roll back to the previous working setup, allowing the IT team to fix the problem. This means the servers will continue to run with little or no downtime, allowing the medical team to continue using its software.
In order to distribute the configuration files, the IT team uses GitHub to store them online; then when internet access is available, the remote servers would simply download the latest version of the files.
A Hybrid Model: Developers use Docker, Servers Use NixOS
Speaking of docker images, one interesting twist in MSF’s setup is to allow the developers to continue using the same toolset as before, without them having to adopt NixOS at all. In order to not disrupt the developers, the IT team makes heavy use of containerization. The developers distribute their software in the form of docker images, which are pushed to a private registry. Meanwhile, the servers running NixOS receive nix files that specify a git repo that has information on which Docker images to pull down and deploy.
This lets MSF use the best of both worlds. Developers can continue working in familiar environments using tools like Docker and GitHub, while the infrastructure team can get the benefits of NixOS on their servers without disrupting the developers. The result: A maintainable, predictable system.
Weekly Updates
Because updating a system running NixOS is as easy as downloading a new set of configuration files, all the MSF IT team needs to do is update their configuration files on their GitHub repository. Every week each one of their deployed servers checks the GitHub repository for updates; if there are changes, the changes are read in from those files, and NixOS immediately applies the needed changes, all without the need for careful monitoring by a local IT member.
While primarily known for its top-notch first response medical care to remote regions of the planet, what’s lesser known is their technical team’s expertise in deploying computers to regions across the planet to help keep the clinics moving with the technology needed for modern medical care.
And because of the vital importance of keeping their systems running and well-managed, NixOS was a natural fit.
Conclusion
MSF, also known as Doctors without Borders, was faced with a technical challenge: How to keep a fleet of computers deployed to the most remote regions of the planet, synchronized and up to date, with little infrastructure and grid to assist them. NixOS was the right fit. It allowed the IT staff to update their configuration files, upload the files to GitHub, and for the remote computers to connect to GitHub, pull down the changes, and make the updates, all with little or no human intervention.
For MSF, the decision to adopt NixOS on all their field servers aligned with their mission to provide fast, optimal medical care in some of the most remote places in the world as well as the need to monitor their infrastructure. Without the luxury of a traditional IT infrastructure, such as server rooms and consistent internet connectivity, they needed a setup that could work reliably without constant attention. NixOS helped make this happen.
While you might not be deploying computers and teams to the harshest and most remote parts of the world, your organization still needs a manageable infrastructure. MSF chose NixOS because it’s an ideal operating system for organizations that need to grow and scale their infrastructure with easy setup and minimal hand-holding and fuss. Whether you’re maintaining a workforce distributed across regions within your country, or an in-house set of laptops, NixOS makes configuring the systems consistently as easy as building a few configuration files. Configuration files can then be stored and saved alongside code in repositories such as GitHub, and updates can be applied easily, with simple rollbacks, although rare, when necessary.
About Numtide
Part of the expertise of the MSF team comes from their partnership with Numtide. We at Numtide worked closely with MSF to make their successful switch to NixOS. At Numtide, we specialize in helping organizations make the transition. We’ve guided humanitarian teams such as those at MSF, research labs, and enterprises alike in adopting reproducible, declarative infrastructure that simply works. Our team played a central role in helping MSF make the transition to NixOS. Ready to learn more about how we can help you transition to NixOS as well? Contact us at [email protected]
Check out the full presentation at FOSDEM here!
[1] https://www.msf.org/donate
[2] https://www.doctorswithoutborders.org/