The Future of SimplyTest.me

SimplyTest.me

PatrickD, the maintainer of SimplyTest.me, recently expressed interest in transitioning the application to a new maintainer. Let me first acknowledge how valuable this tool is for the community. It helps first time contributors who want to rapidly test a patch. It helps community members prototype modules or distributions without the concern of standing up a server. It’s really helped people, especially those not deeply technical, and should continue to as far as I’m concerned. I’ve used this at code sprints, I’ve tested patches on d.o, and generally point people to this in lieu of setting up a formal sandbox. I have been and will continue to be a big fan of this tool and what it does for the community.

I’ve expressed interest in helping maintain this tool moving forward. I want to share my thoughts on my future vision and my interest in this tool. Let me state explicitly that this tool needs to remain free for community members to use, it should remain primarily open source, and modernized for Drupal 8 (and beyond).

My future vision

Initially this project would remain completely as-is and modernized over time. I’m happy to cover the costs and would be even happier to have a hosting company sponsor some servers to migrate the application. Over time, I see some tremendous potential to make improvements to both the infrastructure and the application. I’d love to promote awareness and grow contributions.

The infrastructure should be modernized to support more cloud-friendly technologies like Docker and Kubernetes. This is especially appealing for the rapid provisioning and tear down capabilities container-based technology offers. Tools like Rancher and Prometheus also offer operational benefits that could be considered for a future infrastructure solution.

The application should consider improvements around Drupal 8 best practices. I think the community would have a lot to say on new features. The application should remain as user-friendly as possible, though. There should be a balance of technical awesome with a low barrier of entry. I’d like to see stronger support for Composer (something like Composy, but open source) and offer some advanced features that allow for users to easily export their code/config to Github or even a hosting platform. Since the goal is to remain open source, the application and tools can remain community focused and can accept contributions based on people’s needs.

There has been some discussion of involving the Drupal Association in this decision. I’d be happy to work with them and anyone else in the community to ensure this is transitioned effectively. I’d also be happy to work with companies in the community to help offset the costs of infrastructure, maintenance, and development to keep the platform sustained.

My interests

Only solve the same problem once, they say. There is a space of daily tasks that I perform across all of my projects. I don't want any sort of variation, because consistency begets efficiency. Quality goes up. Learnings are captured with ongoing commits. These are the habits I strive to do and promote. We do the same things all the time. I may want to provision a new site, create a new repository with the same scaffolded code, or have a starting point with an opinionated set of tools, documentation, and processes (Git hooks, architecture documentation, onboarding guide/readme, etc). I want these practices updated with everything I and others learn. This is at the heart of DevOps.

I have solved these issues by creating to be a library of utilities (commands) that could fill team-level operations on a project, and make use of prominent existing open source projects like Drush, Drupal Console, Composer, and much more. I always wanted “the way I work” to be automated and scrutinized by others. That effort grew into the Bild project, which I open sourced. These motivations continue to be my goal with Bild.

To this day, I never really had a focus for Bild. I didn’t promote its existence. I didn’t maintain it like I should have. And, I had a terrible time describing it’s value add. My vision for Bild is basically managing the operations for any Drupal project (not module or theme, but Drupal application operations) and maintaining operational uniformity between development environments and hosting environments. This is it's focus. It took me basically until this holiday break to realize it. And, I believe there is a clear need.

SimplyTest.me is a tremendous opportunity to leverage and reinvigorate Bild. There would be countless lessons learned and efficiencies gained from integrating Bild and the SimplyTest.me application/scripts. SimplyTest.me would then rely on Bild for its library of operational commands and create a robust, user-friendly application that invokes the commands as-needed. The decoupled nature of these tools allow for a great deal of flexibility in use and operation.

A Brief History

When I was at Acquia, we often asked team members to jump between projects to help when there was a need. As a large organization, there were varying levels of consistency across projects, most notably the use of tools tied to the Acquia Cloud platform. I saw a real need for greater consistency, especially in operations. Born was a prototype, developed as a Symfony Console application, that furnished a set of standard commands in this DevOps space. While the idea held merit, some of the engineering managers at Acquia had different implementation ideas. I wanted the tool to be a full DevOps tool that standardized operations throughout the entire project lifecycle, not just a project scaffolding technology (a single platform that could evolve based on project learning). Furthermore, stating a desire to adhere to the least responsibility principle, the use of a more formal "make"-based tool was selected in Phing. There was some existing code already using this approach, so a decision was made to go in a different direction than what I proposed. Due to these creative differences, I lost interest. I really felt that this choice was a major step backwards in terms of where the larger PHP ecosystem was going. I kept using this tool and open sourced the work as the Bild project, yet I never really promoted its existence.

The other build tool at Acquia eventually evolved into Acquia BLT. As I originally envisioned, BLT inevitably dropped Phing and adopted a Symfony Console architecture shared by both Drush and Terminus in Robo. This mirrored feedback I received at CivicActions for a future version of Bild. It's nice to see BLT adopt such a promising framework. While I still had passion for Bild, I couldn't compete with Acquia and I was really struggling to communicate my vision and value add for the tool. It sat for a while.

I eventually met Greg Anderson at Stanford Drupal Camp. He and I sat down for several hours and talked about Bild developed with Robo. I shared what the 0.x branch offered and we got an initial port of Bild on Robo posted to a 1.x branch (which is a work in progress). As of now, much of the commands I had were developed for Drupal 7 and my knowledge of the DevOps space evolved both for Drupal 8 and new container-based provisioning platforms like Lagoon, DDEV, Lando, and Docksal changed the commands which were developed mainly for bare metal hosting platforms. It once again sat, as I didn't have time and I didn't have an actual focus (YAGNI principle). Until now.

Next steps

While companies like Acquia and Pantheon offer PaaS solutions and they open source parts of their code, their platforms still tend to be proprietary. Acquia, specifically, offers single site instances and site factory, which assumes a high level of functional parity across many sites. Many organizations maintain multiple sites, but don't want to pay on a per-site basis. To avoid vendor lock-in, I've always wanted an open source PaaS. A tool like Aegir appears to offer PaaS functionality. I have not explored the Aegir ecosystem, which seems to be very robust and address many problems in the space. I could be wrong, but it appears to work well for bare metal, multisite Drupal applications. I’d love to see a simple site management tool that does not rely on Drupal multisite and can leverage new technology (Docker, Kubernetes, etc) easily. But, admittedly, there is a lot Aegir offers and my main motivation is to explore a fresh and (hopefully) simplified approach to a Drupal PaaS.

My future vision is a PaaS that is totally open sourced and doesn’t reinvent the wheel. I want to leverage Bild as a Robo-based library of utilities and then create a new Bild PaaS application that serves as a lightweight, extensible PaaS application. Maintaining Bild as a separate project helps it have utility outside of the PaaS yet have uniformity within the operations of the PaaS concurrently. Bild PaaS would have a plugin for Bild integration, to leverage all of it's commands within the PaaS. The Bild PaaS plugin system will be initially tested with one of the aforementioned provisioning platforms to ensure it's sufficiently extensible. Bild PaaS could also be extended to support basic PaaS functionality (provisioning, tear down, etc) on any infrastructure (Amazon AWS, bare metal, etc). The current Bild PaaS leverages Symfony Flex and it's recipe-based architecture, which simplistically sets up the plugin systems, leverages object-orientation, and already has a huge ecosystem of readily available tools to extend the application (doctrine, easy admin, logger, debug, etc).

I am currently working to finish the initial Bild PaaS application. The PaaS prototype is in development, will be open sourced once there is an alpha release, and will then be decoupled and serve as the initial set of Bild 1.x commands. I will then review the SimplyTest.me scripts and either create new commands or improve on the existing Bild commands. I would love to have community members try this out, file issues/feature requests, write plugins for different use cases, and help with documentation. Once the initial, prototype PaaS application is done, new Robo commands will be pushed to Bild and invoked as a plugin from the PaaS.

I would also suggest commenting on PatrickD’s simplytest.me issue for its future or give me feedback on my future vision of the project. I would love to help ensure it remains a vital tool for the Drupal community well into the future.