Installer Sprinting at Plone Symposium Midwest
Sven Strack, Paul Roeland, Kurt Bendl and I -- with help from the assembled PSM masses -- got lots done on Plone 5 installers.
A major version change is a great opportunity to shake things up, so we spent the first half day of the conference detailing what we want to do for Plone 5. Paul, Sven and I have been tossing ideas at each other for months. With help from Kurt, it was decision time. In a head-spinning half day, we made all those decisions, then the fingers hit the keyboards.
The next Unified Installer will try to be a bit kinder to new users by allowing them to choose configuration options interactively. You'll be able to use command-line options as before, but if you don't supply any command-line options, we'll start asking questions. When a console-dialog manager like whiptail or dialog is available, we'll use it to gather input. When not, we'll use text and read input from the console.
Steve wrote a wrapper that allows for menu, alert, confirmation, string input and password-input dialogs to be written with the same bash commands. whiptail ships with Ubuntu. dialog is available nearly everywhere else, including via homebrew on Macs.
The UI will also incorporate Sven's dependency checker and installer. Written with help from Giacomo Spettoli, Alexander Pisa and Paul, this tries to sniff out the Unix variant and make the right decisions on system packages. You can have it list out needed dependencies, then make the decision whether to install them yourself or have the program do it.
OS X Installer
Plone has had a GUI-based binary installer for OS X since Plone 2.5. It is, however, getting harder and harder to maintain as Apple continues to make OS X a less and less welcoming environment for GNU-tools-style development. This reached a new extreme when Mavericks' Python didn't work with Mavericks' c compiler without special environment flags that Apple is promising won't work soon.
We've decided to address this by thinking of OS X as a homebrew environment -- in the sense that Ubuntu is a debian-packages environment and CentOS/Redhat/etc. are RPM environments. We'll strip from the Unified Installer the code that's OS X-specific and instead require a system-packages kit available from homebrew. This allows us to socialize the problem of getting GNUish tools and libraries that actually work together.
For Mac users who aren't comfortable at the command line, Sven worked on a set of AppleScript script wrappers that will allow a double-clicky install that does everything from install Apple's current command-line tools to setting up the homebrew kit to downloading and running the Plone installer. (Anyone who suggests that we should have done this in Swift will be ruthlessly ridiculed.)
Windows continues to be a problem for us. Giacomo has been heroically maintaining a high-quality Windows Installer, but he's been doing it for love of Plone rather than because he's a Windows fan. If we're to maintain a Windows installer, we need one or more people or companies that actually use Windows to step up to work on this installer. A great opportunity: Ross Patterson has a proof of concept working for a Plone WebPI (Microsoft Web Platform Installer) installer. But Ross also is not a real Windows user, and doesn't want to take on long-term ownership of the project.
Virtual Machine Images
We believe the importance of traditional installers will be fading with the rise of virtual machine / container deployment. We want to make it point-and-click easy to deploy Plone on AWS, DigitalOcean, OpenStack, Docker, VirtualBox or VMWare with Plone-community-blessed images. We anticipate making images available for testing/deployment and for full-stack production configurations.
We decided on Packer as our fundamental tool for this. Sven's been testing Packer, and has proof-of-concept tests for automated builds of popular image formats. Note that behind the scenes, we'll be using the same provisioner for all the environments.
We had a fun Birds of a Feather breakout from the symposium to talk about repeatable server provisioning. Various folks in the group had experience with Puppet, Chef, Salt and Ansible. CalvinHP and Claytron, in particular, talked about managing lots of VMs with Salt/Minion and experiments with Ansible. Allegiances didn't change, but Paul, Kurt, Sven and I thought Ansible was the clear winner for the installer team's purposes. Deciding points for us were simplicity, use of ssh as a transport, and that it's written in Python.
The Installer Team plans on making Ansible playbooks and roles available in kits that are immediately useful for provisioning everything from testing/development to full-stack production servers. These playbooks will also provision the virtual images we create with Packer.
We hope we'll have lots of community involvement in creation and maintenance of Ansible roles for Plone-oriented deployment of apache/nginx, varnish, haproxy, supervisor and buildout.
The Ansible kits will be a good option for integrators that aren't satisfied with the choices made for the community VM images, but would like to benefit from known-working stack configuration. A typical workflow might be to fork the Plone Ansible kit, customize a configuration file, run the Ansible playbooks to set up a server, and turn on your Plone site. Those specialized integrators will be able to use the kit in conjunction with Packer to create their own VM images.
We continued discussion and research that began in Brasília about a viable replacement for ZopeSkel/Templer, which are at best unevenly maintained and have dependencies on unmaintained code that's not keeping up with changes in Python's setuptools.
We considered a wider set of options, looking for a skeleton generator that's got larger community support, is actively maintained, and meets our basic needs. Claytron did heroic research/testing for us. At the end, only Mr.Bob remained standing. Mr.Bob the skeleton generator, not Mr.Bob the portable-toilet vendor.
Thanks to ...
UW Oshkosh was a great place to sprint, with easy movement from formal classrooms to dorm lounges, to restaurants and parks. Thanks to Kim Nguyen and the UWO crew for planning it all and taking care of us, and to the sprint sponsors: UW Oshkosh Office of the Provost, Six Feet Up, Wildcard Corp. and the Plone Foundation.