people

Weekend Thoughts, 9/11/21: A legacy

admin — Sat, 11 Sep 2021 18:08:01 +0000

Weekend Thoughts, 9/11/21: A legacy

On the 20th anniversary of 9-11, it’s hard to ignore how fleeting life is. What we accomplish during our time is up to us.

What do you want your legacy to be? Are you fulfilling it? Do you maintain awareness toward your goals and what impact you want to have?

While I’m most proud of my family and the legacy I will leave through my kids, we are also afforded opportunities to leave a legacy professionally.

Building great products is an incredible opportunity to help and serve others. The trust and confidence customers place in our hands is not a responsibility we should take lightly. We need to bring urgency and passion to solving their problems and innovating on their behalf. We are trusted to do so.

Investing in open source, and specifically efforts with SimplyTest.me, reminds me that volunteer efforts also leave a legacy. It is important to help create future generations. Everyone deserves the same opportunities I have been afforded. This is especially so for those underserved and underrepresented. While tech can be a difficult and demanding career path, it is our responsibility to create opportunities for those to learn, grow, make a living, and also have impact. Open source is critical to lowering barriers and bringing an equitable, accessible foundation to build upon. All ships rise.

Covid-19 has brought new found awareness and urgency for many to fulfill their goals in life. Many have chosen to chase their dreams, try new careers, and test an employers market at the chance of a better path ahead. While the world re-opens, many are not turning back to the jobs which didn’t have a good culture or held them back from earning a reasonable living. Employees have a ton of opportunities right now and employers need to evolve. It is overdue that our priorities evolve in such a way. Businesses doing so are thriving and many studies demonstrate that a good culture retain employees and the employees are actually more productive. It’s an environment that helps people thrive. I want this to be a cornerstone of my legacy.

During this somber day, I hope and wish we all do our part to find and foster opportunities to establish a new and more equitable future, demand better work environments and work culture, and help ensure all people reach their goals during their time on the earth.

Finally, don’t forget to recognize and celebrate impact when you see it. Recognize the accomplishments of your teams for the amazing work they do for customers. Cherish people who show up every day fighting for a better culture and the impact you want to see. And celebrate those who reach their goals both at and even outside of your company. Spread more good news than bad and drive the kind of culture that isn’t purely about business objectives but one that raises up the people that help businesses achieve them. People are, in fact, the most important thing a business has.

admin Sat, 09/11/2021 - 14:08

Tags

Weekend Thoughts, 9/4/21: Leading through change

admin — Sat, 11 Sep 2021 18:05:11 +0000

Weekend Thoughts, 9/4/21: Leading through change

Changes to teams, organization, structure, or process should be thoughtful and done with patience. Change can take time to materialize. Teams often don’t like change. Making changes too frequently often doesn’t allow a team to get past the storming, norming and performing phases and therefore doesn’t allow for the expected investment to be realized.

And, changes are disruptive. Changes should only be executed when it’s necessary. They shouldn’t be knee jerk or reactionary. It needs to be well thought out and planned. Slow and methodical.

Plus, we are all human. Change can be hard, even for those leading it. It can throw people off. It takes time to build trust after. People may leave or be frustrated. Looking past the human element demonstrates a lack of empathy. Good leaders lean into an opportunity to strengthen connection with those impacted. Listen with empathy. Look for opportunities to make further adjustments based on what you hear and learn.

And finally, remind those involved in changes of their value to the organization. Changes often position people to have a different, but significant impact. Explain the “why” and how that impact allows the organization to be better.

admin Sat, 09/11/2021 - 14:05

Tags

SimplyTest.me Welcomes Matt Glaman

admin — Sat, 02 Jan 2021 19:28:08 +0000

SimplyTest.me Welcomes Matt Glaman

SimplyTest.me is a labor of love but it has been one of the most unique, satisfying contributions I've made to the community. Most of that labor has been dedicated toward maintaining the Drupal 7 version. Our maintenance was massively improved with the introduction of Tugboat.QA, but this year found major changes in the Drupal community with the launch of Drupal 9, the major changes to Composer workflows, and the Gitlab integration. All of these things are of high impact for Drupal and required major changes to SimplyTest. This has been especially hard to keep up with for me. Our latest incident showed the limitations of the current system, where SimplyTest.me was blocked from the Gitlab infrastructure due to updates in Drupal.org's patching workflow. Much of this pulled my attention away from our aspirations to launch a new version. Until very recently, that is.

On a whim, I had asked Matt Glaman if he could help. He had a little bit of time over the holiday season to pitch in and this rapidly changed the course of our new implementation. Matt stands out for his deep knowledge of React combined with his knowledge of Drupal itself. Several people have pitched in on the React frontend, but it was incomplete. The backend was also only partially completed, difficult to know what was left given it wasn't yet integrated, and was largely based on Drupal 7 code with only nominal improvement. Since we implemented Drupal 7 last year, Tugboat itself made some nice improvements that also were not implemented yet. In SimplyTest, we had the chance to simplify the architecture by more direct API calls and removal of the repository-based approach we used. React itself had an opportunity to invoke Tugboat APIs in a more straight forward way for logs, progress updates, and more. In short: we needed Matt.

Matt graciously agreed to help and at a community rate. We will be sponsoring his time through the Open Collective. Any contributions to SimplyTest's Open Collective will help us finish this effort up faster.

Matt is an expert. He knew every part of this stack. We needed someone to come in, sort things out, and get the basics of things in place to help move this effort forward. I was able to provide answers on what needed to happen, but I wanted Matt to have the full autonomy to implement it properly. This investment now delivers brand new long requested features, sets up future contribution in new ways, helps us make huge strides toward shutting down our Drupal 7 current system, and is overwhelmingly simple.

I want to thank Matt for making this possible. His contributions have brought a much needed jolt of energy into these new efforts. I can't wait to share this work with the community. Thanks for helping the cause, Matt!

admin Sat, 01/02/2021 - 14:28

Tags

Finally! A website refresh

admin — Tue, 29 Dec 2020 03:43:46 +0000

Finally! A website refresh

Let me welcome you to the new nerdstein.net. I'd like to thank Ana Laura Coto for help with the design. I'd like also thank Jonathan Daggerhart for being there when I needed an assist. I am really happy to finally share this project with the world.

This has been a long time coming. Three years. Yes, you read that correctly. Three years. Why? I prioritized learning. And, I am slow and methodical. This website is a culmination of many things I learned over a three year span.

But, three years?!? Yep. I have a rewarding and challenging full time job at Acquia. I have a family with two young kids. This project was all done in my spare time and at my convenience. Most of my time and learning is invested in SimplyTest.me. But, this project gave me a high degree of freedom on my own timeline. I wanted to learn things well and understand things properly. I did it piece by piece. Drupal evolved a lot in three years, as well. There was a lot to learn. And, then 2020 happened (I'll hopefully blog more on that soon).

For the rest of this blog post, I'd like to reflect on these efforts and learning.

From Drupal 7 to Drupal 8 and then Drupal 9

Drupal evolved significantly from Drupal 7 to Drupal 9. My original goal was to launch this with Drupal 8. Then life happened. Then Drupal rolled more features and made it even easier and better to launch my site. I'll share some specifics below. Regardless, Drupal has established itself as a great tool for progressive enhancements.

One notable change on this journey is how the community adopted Composer. I recall the very basic support when Drupal 8 was launched. Major changes were made when I upgraded the site to Drupal 8.7, which totally revamped the Composer work. I did the same when Drupal 9 came out. The community made some significant progress, but I'd love to see this stabilize now. I made heavy use of Composer and it's DevOps capabilities to script some of the integrations with my design system. This area of work was not well documented, especially in moving between versions and resolving issues.

New server

I was long overdue for a tech refresh. I had the same VPS running my Drupal 7 site for over seven years. It served me well. I spun up a VPS for a few Drupal 8 sites I hosted for local non-profit orgs that couldn't afford it. My Drupal 8 development site ran there for several years. The non-profits moved on due to grant funding and poor technical support (hint: if you can't do something well, don't do it). And, that Drupal 8 server rapidly dated itself.

I was able to shut down both VPS servers and create a brand new Ubuntu 20.04 on Linode. Linode has been a sponsor of SimplyTest.me, helping offset the cost of it's development infrastructure. I'd hardly call the experience of Ubuntu 20.04 learning; it was a breeze setting up a Drupal 9 environment. These systems have come a long way even in seven years. Shutting down the VPS servers is bittersweet. I've removed some significant tech debt but those servers were rock solid. No regerts.

Design Systems and Drupal Integrations

The biggest emphasis of learning was the front-end work of the design system. I am still so far from an expert. But, I definitely understand it now. I started with Pattern Lab. I broke down Ana Laura's design into a series of atomic components. I developed them. I learned more about SCSS and building of front-end assets. I mocked up these patterns with JSON data objects and finally built out representative pages with the pattern.

Once the design system was done, I started on the Drupal integration. My goal was to pull as much as possible from the design system (no cheating!). Composer made it easy for me to keep the design system and theme separated. I leveraged the Components module and proceeded to site build and theme simultaneously. Some of this was challenging. It was a fight between properly structuring content, overriding Twig themes, leveraging the Components module to include pattern markup, and parsing Drupal-related data objects.

I blogged about this several times, both in terms of the concepts I used and the architecture I leveraged. Please feel free to read more about this if you are interested.

Twig. All the Twig.

Developing Twig was hands-down the most time consuming part of this project unquestionably. I spent countless hours digging through Drupal rendered objects, raw data, attributes of that data, and much more. I tweaked the display settings, created new view modes, did more tweaks, changed more Twig, and this went on endlessly. Data objects passed to Twig were massive and nearly impossible to debug without printing keys, refreshing, and traversing deeply nested objects. The syntax changed with every different entity and then even more with Views. Yes, Drupal is incredibly powerful and extensible. But, I continue to be surprised by the developer experience around Twig. I just wish this was simpler. I know the community has developed several utility modules. I need to dig into those more.

Media System: image links

I had aspirations of contributing a composite field type module back to Drupal core. Developing this module, which never got complete, was a huge rabbit hole. I thought it would be relatively straight forward to inherit the core-provided image and link field types into one new field type. This was significantly more complex, especially when dealing with cardinality, managing data storage/schemas, and dealing with field widgets that were built atomically. This approach was rapidly becoming complex and it didn't seem like it needed to be. Then, help was on the way...

Daggerhart shared with me the new Media entity delivered as part of the Media system in Drupal 8. I was able to create an Image Link media type that was composed of an image field and a link field. This was super easy. From this, I got all of the expected benefits: Views support, easy integration with other entities, and the perks of the new media features in Drupal. Lesson learned: spend some time figuring out what parts of Drupal can be used before diving into code.

Migration from Drupal 7

My website is quite simple. It's a basic blog. I wanted a simple way to migrate the content. Drupal offered that with it's native Drupal to Drupal migrate UI.

Set up was simple. I was able to set up a secondary database within settings.php for my Drupal 7 database. I archived to the files directory on Drupal 7, extracted them in a specific file path, and entered that path in the UI.

When I ran it, I got a nice confirmation page of potential issues with the Drupal 7 configuration, modules, and structure. I worked through it, ran the migration, and it basically worked. I had a few nodes that did not properly map their type. I cleaned this up through the database by manually editing the node type in the node and node revisions tables, followed up with a cache clear. I also had a few nodes that did not display the body field content. I simply re-saved the node and it worked.

I had to run this migration multiple times over three years as I refreshed the content on my current site. The UI didn't offer a rollback (this would be a nice feature through the UI) and had a big red note of caution when running multiple migrations. I did it anyways and it seemed to work fine for me (again, with my simple use case). Buyer beware.

To learn what happened behind the scenes, I audited all of the migration classes and configuration that were created. Kudos to those who worked on making the migration experience a positive one. I was really surprised how much was automated, how insightful it was, how smooth it went, and how easy it would have been to override the underlying config it created.

TugboatQA

As I previously shared, I wanted a tech refresh. I thought about spinning up a development server, but this isn't really a modern CI/CD workflow. Having such a positive experience with TugboatQA on SimplyTest.me, I wanted to rapidly see changes before posting them live. And, I wanted something that didn't require me to mess around with another server. TugboatQA for the win.

Set up was straightforward. This may be potentially because I understood most of the concepts from SimplyTest.me, even though the use case couldn't be more different for my personal website. I followed the documentation and configured it within a few hours (love their IaC set up). I actually found a bug in how I implemented Composer paths, which I subsequently fixed.

Ported Block Type Templates module to Drupal 9

Again, I spend a lot of my personal time on SimplyTest.me. I still moonlight with some module maintenance, but I do a pretty poor job at this if I am being honest. I did use the Block Type Templates module I developed to have more fine grain control of block Twig. This was immensely helpful with the design system integration. But, I had to port it to Drupal 9. I also cleaned up the issue queue and added some nice features. I hope this helps others, too.

Future Goals

This website is definitely not done. I have some open issues I've logged. I need to clean up some content, fix even more styling, and finish the Tugboat implementation now that I've launched. It's done enough for me to not maintain two websites any longer. Please feel free to let me know if you find any issues.

I also want to add more DevOps. When a PR gets merged, I want to set up a GitHub action to trigger the deployment on prod: pull updates from Git, load the config, run the database updates, and clear the cache. My confidence is a lot higher with Tugboat and my CI/CD workflow in the mix. This should be fun.

I'll be finally adding SSL support for my site, but I'd like to do this by learning more about the edge layer. I need more research but leaning towards using CloudFlare for this.

Pattern Lab has evolved majorly since I first set up the design system. I'll be upgrading this with time and I have the Drupal integration versioned to allow for changes like this to be done in the future.

Final Thoughts

I'm glad to finally be able to launch this thing. I learned a lot doing this. Drupal is still a fantastic option for digital experiences. It does so much. Drupal 9 brings all of the previous years of learnings, new features, great performance, modernized dev workflows, and continues to deliver a highly extensible framework. I thought a lot about developing this in something like Gatsby, but would have still wanted a FOSS backend like Drupal. It did everything I needed natively.

admin Mon, 12/28/2020 - 22:43

Tags

Drupal Community Care Packages

admin — Thu, 30 Apr 2020 20:33:00 +0000

Drupal Community Care Packages

Our community has always meant more than just code. These last several months have been difficult to make the same kind of connection we have come to expect from things like in-person events. And, during this time, many of us are having to juggle new challenges, may have sick loved ones, face professional or financial uncertainty, and more. I’m choosing to do something about it.

I am announcing free community care packages for Drupal. I care about our community and I care about the wellbeing of all of you. I recognize this is basically nothing at a time when people are dealing with much larger issues, but I hope that even a small gesture can help to remind you that you matter.

I’m extending the concept of the coffee exchange. What will be included? One-half pound of coffee (thanks to my friends at Standing Stone for help with the coffee), some stickers, and some local chocolate (thanks to my friends at MarCia's Chocolates for suggestions and help with shipping). Shipped to your door. For free. This is my way of saying thanks and recognizing what I miss the most from our pandemic times: each of you.

For financial reasons, I need to keep this within the United States and to the first 30 participants. I will try to do more, if possible. I encourage others to steal this idea, especially for their own country or area. And, I would gladly accept donations to expand to a greater number of people.

What do you need to do? Nothing. However, I’d love to hear your story, if you are inclined to share. I’ll release people’s stories, for those who opt-in, in a series of community spotlights on my blog to help promote the kind of connection our community affords. I believe we need these types of messages to balance the difficult and challenging news we see each day.

How do you sign up? Simply fill out the form here. I’ll remove people’s personal data once the packages get sent, which will hopefully be within two weeks.

admin Thu, 04/30/2020 - 16:33

Tags

A DevOps Primer

admin — Tue, 30 Apr 2019 02:06:10 +0000

A DevOps Primer

I just closed about 100 browser tabs from an early year activity. While it’s embarrassing I left those tabs open so long (going on five months), I wanted to leave them open to reflect on what I learned. And, what a better way to reflect than a blog post.

The Soapbox

Bear with me for a moment. I subscribe to the philosophy that DevOps represents a line of thinking and a way of working, not a technical product. Like security, when someone says, “I’ll sell you DevOps”, it’s a scam. For those “doing” it right, you need to focus on foundational concepts built into continuous learning. This ideology has been communicated in talks and blog posts by some of our communities’ sharpest minds, including Michelle Krejci, Jeff Geerling, Mark Schwartz, and more. And, in my opinion, there is a direct correlation to Agile frameworks. Concepts like retrospectives to share learnings, sprint planning to prioritize, and backlogs to organize and track work thoughtfully. Most of these ceremonies are open to the whole team to participate, so they naturally advocate for transparency and connection.

Key point: DevOps is ongoing evolution through experience. Technical progress is just one type of outcome, but equally important are solutions involving people and process.

DevOps Philosophy

Change is at the heart of DevOps. Digital products change. People change. Organizations change. DevOps reflects a pragmatic approach to change. Many people discuss it, especially solutions. But, any solution reflects a change that is, hopefully, driven by what is learned and what can deliver better, more impactful results for actual people and real problems encountered.

As a basis, start by evaluating what change occurs. On the surface, change can look like any of the following:

System and software updates, especially security ones
Continuous Integration (deployments of change increments, automated/manual processes)
Changes in the team (people leaving, people being promoted to new roles)
Organizational changes (launching a new product, leadership transitions, exploring new markets)
Incidents (getting hacked, services going down)
Exploring new technology
Sunsetting legacy technology

A mature DevOps practice is capable of adapting to change ongoing and remaining effective. An equally important philosophy is the openness and willingness to change. This is built on trust and the recognition that every team member is capable of bringing value: to allow people to raise their voices, drive change, and have impact.

DevOps can be transformational. It can be really hard and a non-natural fit for the “command and control” models traditionally established with embedded IT departments and IT vendors. Mark Schwartz writes about this concept routinely in blog posts and in his book “A Seat at the Table: IT Leadership in the Age of Agility.” Again, organizations suggesting they can just buy DevOps would be in for a rude awakening or really only purchase an incremental step relevant at that point in time only. Sure, you can purchase a two million dollar Kubernetes, cloud-based infrastructure framed as DevOps. But, what problem are you really trying to solve? Chances are, you’ll be in a position of opening up countless other problems around vendor lock-in, governance, maintenance, and enablement you never anticipated. A better place to start is in the cultural change needed to identify problems, define risk, share ideas and make smarter investments capable of having real impact now and in the future.

Bare Minimum DevOps

Now that I’ve set the context for my thoughts on what DevOps actually is, how do you do it? I’ve long wanted to make a blog post defining some bare minimum DevOps concepts. I’ve seen so many differences when I interact with a client or project. I’ve wanted to point back to something and highlight my thoughts foundationally. The range of differences reflect the varying states of learning organizations are at. And, it’s natural given many of the concepts represent years of learning how to build quality, robust digital solutions that are capable of adapting to change.

Case Study: SimplyTest.me

Remember the browser tabs and the learning I did? I was working on SimplyTest.me. As of right now, it is a perfect example of a system desperately needing some DevOps love. There has been a lot learning and needs are defined. As the project lead, I’ve long wanted to automate routine failures, establish a more easy-to-change system, and open up SimplyTest.me for broader contribution. But, the current system is very hard to do so as it’s developed. The infrastructure is highly customized, the tool is fairly complex, and, subsequently this makes it harder to change. This may make for a future blog post, but the concepts shared reflect this recent exploration for SimplyTest.me and things I’ve encountered over the years.

Environments

Change cannot and should not happen only on production systems. Any change needs prepared, tested, and then deployed to prod. At a minimum, you need both production and non-production environments. Change increments then get vetted through non-production environments before being released to production. A non-production deployment helps vet any changes made on a local system before they hit production.

Parity

Parity is a key concept for evaluating change. As an example, if you are running different versions of PHP across environments, evaluating a change increment on one environment may not match the results of what gets deployed to production.

Evaluation Criteria

If you change things, you need to establish a means for evaluating the change. This could be both manual or automated testing of a change. It could also be part of your development workflow on what needs to be tested when implementing a change. But, the key is you need to have some process defined. And, this process needs to respect both the change increment itself and some means of looking at regressions throughout a system. Because, any change made in one part of the system can unknowingly impact the rest of the system. Establishing some criterion to evaluate change may be just as important as the change itself.

This is not confined to technical change either. When making process or organizational change, there should be some metrics and data used to measure efficacy. After all, if you are making a change, how do you really know if the change worked?

Source of Truth

Also known as the “system of record”, a source of truth is a data integrity concept. Some aspects of environments are more flexible to change than others. Most production systems maintain data (users, content, files, etc) that most likely should not change as change increments are deployed. And, non-production systems leverage production systems as the source of truth, often by synchronizing or replicating a “point in time” snapshot of the data to mirror production data before a change is evaluated. The source of truth is left untouched until the change is ready and often throughout a deployment of a change. Third party systems may be the source of truth for other enterprise functions. For example, an email server may be the source of truth for notifications sent from a system. Recognizing the differences and responsibilities of specific systems and environments help understand source(s) of truth and how they serve a larger change process.

Backups

Even with the best of intentions, changes can break things. Be prepared at any point to restore from a backup, both by rolling back changes and by restoring databases. Test this process (really… it needs to work in the most inopportune of times). Having timely and relevant backups are hugely important for restoring botched deployments, standing up systems after a security incident, and generally gives people piece of mind. Make sure you understand how to restore a backup and bonus points if both the backup and restoration processes are automated.

Keys

Identity management is a really deep and hard problem in IT. On the surface, people think identities apply only to users. But, systems need a proper way to be identified and subsequently verify their authenticity. Keys are a widely adopted approach. Often there is a public and private key, where the public key is referenced by other interacting systems. The private key is used as the verification and is owned strictly by the system. Maintaining keys is a critical way to ensure only specific systems are capable of accessing and associating a system to another (often through users and their specific permissions).

Secured Channels

When systems are interacting, one needs to be concerned with the channels in which systems communicate. Secured channels help encrypt communication. The most common of which is the use of SSL and certificates. Tools like LetsEncrypt have brought SSL to the masses, allowing for a free way to generate (and re-generate) SSL certificates capable of happening in an automated fashion. This helps mitigate the risk of someone intercepting and browsing your traffic between systems.

Scripting

One key way people apply what they have learned is through scripting things that happen routinely. Scripting also can be as simple as a Bash script to perform a routine operation in a consistent fashion or a robust development framework with conditional scripts executing routinely and on-demand. Those intimidated by scripting can start off basically by leveraging tried and true bash scripts that run non-interactive commands across any environment. You can get a long way without introducing complexities found in more robust frameworks.

Code Repositories

Code repositories are the cornerstone of DevOps. They are the source of truth for all system code, any scripts that run, and can be critical in any disaster recovery/restoration process. Code repositories maintain a record of changes and the users who made the change. Code repositories can be cloned to any environment and fetched as needed (the “D” in “DCVS”). Concepts like branches allow for changes to be staged before being merged into a branch associated with production environments. Releases can be implemented as tags that reference specific commits. And, many popular tools (Github, Gitlab) offer features, like pull/merge requests, that are advantageous within a development workflow.

Also, code repositories maintain change events. Creating a new branch may provision a new environment (this has been made popular by CI/CD models) that allow for hands-on testing of changes before release. Merging a branch can destroy an environment and trigger a deployment to a production or non-production environment. And, tools like pull/merge requests allow for code to be reviewed. All of these events are exposed to a broader opportunity that DevOps can leverage. For example, running automated tests can occur when commits are pushed up to a branch or when a deployment to a non-production environment occurs. None of this is possible without a code repository.

SSH

When events occur and scripts need to be run, how do they run on the actual systems? SSH is the answer. SSH is the protocol used to remotely execute commands from one machine to another. I remember the days where every company had huge in-house server closets with expensive terminals to manage the servers. Now, we can purchase affordable infrastructure managed remotely, upload keys, and open terminal sessions that are directly connected to these remote machines. SSH has made much of this possible and is critical for DevOps.

Jenkins

Continuous integration (CI) tools exist to centralize and execute what you have automated. I questioned if this needed to be on the “bare minimum” list. Inevitably, you can do a lot without such a tool, but you won’t get the mileage you could otherwise. CI tools do a great job of logging execution and maintaining a history of execution. Imagine developers having to save their terminal history. Such tools help teams have a central platform where multiple users can log in, access can be control, parameters can be selected to use common scripts, and more. Tools like Jenkins, Travis CI, and CircleCI have native integration with code repositories, can help manage keys and servers, and can manage complex, conditional workflows.

Systems Infrastructure

All of the aforementioned concepts rely on an infrastructure that runs the systems. At a minimum you need at least one server for production and seperate server(s) for non-production infrastructure/environments. Backups should be maintained offline and outside of the servers.

Infrastructure as Code

While it’s not “bare minimum,” container-based, cloud technology can create a flexible infrastructure capable of provisioning and tearing down servers when change events occur. This maintains a durable, extensible infrastructure capable of responding to change. Tools like Docker can help create the definition and images needed for containers. Cloud-based platforms have APIs for creating instances as needed, managed volumes for persistent storage (logs, databases, objects/files), and monitoring needed for auto-scaling. Production infrastructure, or specific aspects of it, should be persistent. Load balancers and proxying become common when implementing this type of infrastructure, including DNS management. Again, while this may not be completely necessary, it’s certainly an appealing option for those building something from the ground up or people looking to level up an existing infrastructure.

Documentation

This may seem commonsensical, but there must be documentation. At a minimum, I look for a README in the code repository, describing how to set up a project, a technical architecture overview, a description of the infrastructure, and a brief rundown of what is found in the code repository. Non-technical users often need Google Docs or something outside of the technical repository. The most common things I’ve seen are flow charts articulating the DevOps workflows, diagrams of the systems, and high-level commentary on insight and motivations. After all, changes in teams or members on teams can be caught completely flat footed without good documentation.

Summary

As I shared, I think DevOps is sometimes a misrepresented topic because I believe it starts with a way of thinking, is often associated with transformation, and is not strictly about technical outcomes. I’ve seen variations in maturity and tried to share what I consider to be a bare minimum set of concepts people need to both practice and apply DevOps. I can’t imagine the number of browser tabs I’ve opened over the years researching this topic. Again, it’s simply not enough to purchase a product and expect the potential outcomes. What do you consider DevOps to be? What set of bare minimum concepts did I miss?

admin Mon, 04/29/2019 - 22:06

Tags

SimplyTest.me and Google Summer of Code 2019

admin — Thu, 07 Mar 2019 18:06:34 +0000

SimplyTest.me and Google Summer of Code 2019

SimplyTest.me is a project I continue to lead and volunteer my time to. I’m driven to help lower the barrier to entry for people in our community to contribute, use Drupal, and be a part of our community.

For me to be effective, I need to keep my eyes open for unique ways to help the project. Much like our larger community, we must be opportunistic to remain sustainable. Google Summer of Code (GSoC) is just that. I previously participated as a mentor, helping students port security-related modules to Drupal 8. I was approached by Matthew Lechleider (slurpee) to see if there would be a fit for SimplyTest.me. And while SimplyTest.me is unique in that it is both an open source project, a set of services, and associated infrastructure, it has long been a desire of mine to have more people contributing and have SimplyTest open up career opportunities for participants. GSoC is just that.

According to the GSoC timeline, student candidates are participating in the application period. For Drupal, specifically, they are not only evaluating project ideas but getting comfortable with community standards and contribution. Candidates then write up proposals for the GSoC leads and project maintainers to evaluate. Matthew summarized this perfectly by saying,

GSoC can be similar to a "choose your own adventure" type summer job. We provide cool bosses (mentors), a software platform with a request (Drupal/SimplyTest.me), and the option for student to build their own project plan then execute project while being paid via Google.

But, given the fairly niche nature of SimplyTest, a simple project description is hard to develop a proposal. This blog post helps expand on the project description to help clarify potential next steps.

What does SimplyTest offer?

Any student participating in this GSoC project would get a much broader experience than other projects potentially offer. And, while we have a current Drupal 7 based system with a highly customized infrastructure, this project will emphasize our future roadmap in a fresh, new system. This features the following modernized architecture:

A Drupal 8 distribution
Integration with Drupal.org sponsored web service APIs
A React.JS front end application
DevOps efforts (Docker, web service APIs, infrastructure as code)
Integration with Tugboat QA (this is still formally TBD, but a proof of concept in the old system is near completion)

Project ideas

I want to stress that we need to make progress on the new system, not the current one in place. The following serve as some high-level ideas and “current state” of known needs. I will stress that a successful project proposal should reflect the necessary research and details needed to achieve the following ideas. The links below offer further details and insights helpful for an informed project plan.

Implement Tugboat QA - The proof of concept is in the current Drupal 7 system. A Drupal 8-based equivalent will need to be developed. This may include porting the Tugboat QA module to Drupal 8. Students should research this platform and the current proof of concept to get an idea of the work needed.
React.JS front-end - The distribution will have a series of API endpoints which cache data from Drupal.org. These services may need built or enhanced to work with the new React.JS front-end, which is in progress but will likely need to be finished. Students should look at the work-in-progress Drupal 8 code base for an idea on current state.
Implementing the Drupal 8 theme - Our distribution already has a Drupal 8 theme in place. However, it needs to be implemented with the React.JS front-end and applied more generally throughout the Drupal system (e.g. menus, header, footer, FAQ page). Students should look at the work-in-progress Drupal 8 code base for remaining efforts.
Feature parity - There are many Drupal 7 specific features that would need to be ported for feature parity. Some current features are no longer required given the Tugboat QA integration. Students should evaluate the Drupal 7 distribution and articulate missing, but relevant features that can be accomplished in the project.
Analytics - New mechanisms will need to be in place to develop various usage-based reports from within Drupal 8. Students can share potential approaches to achieve this goal.

Please note that once we get a stable set of features, the Github repository will be moved to a 8.x branch on Drupal.org.

Conclusion

I look forward to seeing student proposals and am more than confident SimplyTest.me provides both a level of rigor and educational opportunities. I am also interested in potentially splitting this up into two projects: front-end and backend/DevOps. Any interested students can reach out to me via my contact page. Other potential mentors have been identified that have worked on SimplyTest.me and discussions are ongoing.

admin Thu, 03/07/2019 - 13:06

Tags

My 2019 Aaron Winborn Award Nomination

admin — Wed, 06 Feb 2019 14:25:23 +0000

My 2019 Aaron Winborn Award Nomination

Last year, I read Jim Birch’s blog post outlining his Aaron Winborn nomination of (now) award winning Kevin Thull. As a community, we need to do our best to celebrate accomplishments and I love that this award exists. We need to tell more of our stories regardless of who gets the award. I see no reason not to share my nomination for this year’s award publicly.

Contribution takes many different forms. Our community recognizes many aspects of contribution, but I believe the spirit of the Winborn award is to “think bigger”. Community members find creative ways to have impact well beyond just commits. My nomination tries to honor this with two primary criteria: contribution and impact.

I would like to nominate Nikhil Deshpande and Kendra Skeene. Yes - two people. But, they have been an inseparable force that has greatly impacted the community.

Why? They own, recognize, and prioritize their collective purpose. They serve citizens through the Digital Services in State of Georgia. Everything they say and do, every investment made, every email sent is focused on the needs of all citizens or to better empower those serving citizens. Nikhil and Kendra emphasize the need for research, data, and analytics to drive citizen-first initiatives in usability and accessibility. They approach their roles with empathy. These efforts are pioneering in the public sector -- helping modernize offerings throughout the State of GA and set a model for other organizations in the public sector.

And, they primarily invest in open source technologies. Drupal serves as a vehicle they adopted to deliver many of their digital services. Their efforts have had a major impact to the Drupal community and beyond. Their team has led efforts that have pushed innovation in our community. “Ask GeorgiaGov” connects conversational interfaces like Alexa to Drupal. The State’s Open Data portals integrated rich data visualization libraries through dashboards built from the Drupal distribution DKAN. They invest in projects and hire agencies in our community to help their team. This subsequently creates jobs and new opportunities for community members. Agencies are asked to contribute back to the community as much as possible, ensuring their investments are able to be used by everyone.

What is the impact of this effort? Drupal, as a product, has innovated in new and novel ways. The tool is better able to serve all people. The Drupal community is better sustained, especially those that prioritize giving back. Other digital agencies in the government look up to and model their efforts, which drives even more impact for our community. And, people throughout the world, especially the citizens of Georgia, benefit from their refreshing investment in forward-thinking innovation.

This impact has received wide praise and their approach has spread. Nikhil and Kendra are routinely recognized [1][2][3][4] and speak at both community and public sector events. They are thought leaders that advocate for their “citizen and open source first” approach.

Preston So, Director of Research and Innovation at Acquia, captures this perfectly in the following quote:

Nikhil and Kendra, and the entire Digital Services Georgia team, have been instrumental not only in advocating for open source across state and local government but also in fostering innovation in the Drupal community. Thanks to the forward-looking vision the Digital Services Georgia team has articulated over the last few years, we at Acquia Labs were able to hew to our mission of encouraging open innovation by contributing back to and maintaining the Alexa module available for Drupal 7 and 8 today. This integration has inspired governments and organizations the world over. But it is Nikhil and Kendra's commitment to open source and deep understanding of our community that truly shine a spotlight on their efforts to lock open governments worldwide and make open source just as much a first-class citizen as the Georgians they serve with devotion every day.

Both Nikhil and Kendra have done amazing things for our community and opened up new opportunities in the public sector. Imagine the scale of potential impact they would have to our community if even a few more agencies follow their lead. I thank you both for the example you’ve set, your contributions, and for leading the way in this area.

admin Wed, 02/06/2019 - 09:25

Tags

The Season of SimplyTest

admin — Fri, 18 Jan 2019 19:56:53 +0000

The Season of SimplyTest

Last year was spent primarily learning about SimplyTest. We did make some progress, but I think “keeping the lights on” for a system of this complexity was quite a feat after the project transfer. It’s a unique and fairly complex endeavor that bridges all elements of an open source project, a completely free service, and underlying infrastructure. I see all of the good and the bad that comes from each aspect: system maintenance, feedback from community members, customer service (Slack, Twitter, etc), system outages, and more. I recognize how valuable this service is to the community and I strive to offer the best service possible.

I serve as the SimplyTest lead. I do most things and do not have a team. I steer both the current system (primarily for stability) and maintain the vision to help modernize, simplify, and scale SimplyTest with time. I balance my time by prioritizing impact, knowing I work full time and have a family. There are times progress stalls given time constraints and this seems to mirror much of the concerns people raise around a perceived “stagnant” system.

There isn’t vibrant additional contribution to any aspect of SimplyTest. But, I do want to recognize and thank Greg Boggs for his help, especially with system administration. Iris Ibekwe, who developed a new Drupal 8 theme for our future front-end. Several members of my team (Jonathan Daggerhart, Ryan Bateman) at Hook 42 have helped start build out aspects of the Drupal 8 front-end with me. I welcome help and also need to do a better job of asking for it.

There are little to no finances tied to SimplyTest. We are fortunate to have server-level sponsors (shout out to Maloon, Druid, and Linode), but we must make use of free services/tools, pay personally out of pocket, or (very recently) get reimbursed through donations. All labor has been volunteered and some sponsored by Hook 42 (thank you so much!!). I personally have paid for a new design and logo, stickers, domains, and more, long before the Open Collective was established.

At the end of the year, I often take time to work on outside things (e.g. professional development or community work). 2018 was no exception. I had a full backlog of items I wanted to tackle with SimplyTest and wanted to share the last two months of progress.

Development server

Given the underlying complexity of the various distributed systems involved, it is incredibly hard for people to set things up on their own. This hurts contribution.

Thanks to a generous donation from Linode, we received credit on their platform for new infrastructure. This allowed us to set up both a front-end server and a worker server to be used for development. While this is quite close to being finished, this is a huge milestone for SimplyTest to ensure we vet changes on non-production systems and foster an environment that is safe to make changes such that others can learn the system to contribute.

Clean up procedures

The setup of the new development servers required snapshots and images of the underlying infrastructure containers. Just the action of creating these caused the system to go down due to a lack of storage. This uncovered an underlying issue where spawned instances were not being properly cleaned out and the subsequent images were massive.

Processes were developed to manually clean these out. Based on an increase in usage, the system subsequently went down several other times. This was then scripted and moved into daily cron. We are actively monitoring the disk utilization to ensure these processes hold and stabilize the system.

Continuous integration

A brand new Jenkins server was set up to afford opportunities to automate tasks in and between servers. Attempts were made to automate the creation of the development servers. But, there were many complications encountered, like the previous wildcard certificate used for the instances that was not scripted through LetsEncrypt. The storage issues ultimately prevented the automation from moving forward and the amount of technical complexity to make this a reality with the current system is likely not worth pursuing a complete solution. However, it’s nice to have this tool in place to move forward and maintain a platform for learning and rolling in automated improvements.

New SSL for the web frontend

During the creation of the development servers, the scripts leveraged to generate SSL certificates through LetsEncrypt were executed on the development servers multiple times due to both networking issues and updated configuration. Cleanup was required to remove some failed past attempts on the production system tied to certificate renewal. All of this work required the LetsEncrypt service to be invoked multiple times in a development-only capacity.

Sadly, this effort caused us to hit the LetsEncrypt rate limit. Within a few days, the SimplyTest.me SSL certificate expired (at the most inopportune of times). While it required funding, this ended up being an opportunity for us to purchase and place a more permanent SSL certificate to minimize maintenance moving forward (LetsEncrypt has quarterly certificates). This was our first use of the expense system in our Open Collective to reconcile and publicly share the expense.

Monitoring

Given the expansion of infrastructure, it’s imperative to have monitoring in place. As a start, we’re leveraging the free offering of Uptime Robot, which now covers outages for all production, development, and CI infrastructure. There are opportunities over time to expand this for storage thresholds, proactive SSL certificate notifications, SSL expirations, and more.

Expanded donations

We’ve not done a great job of marketing our Open Collective because I believe firmly that people need to be informed of where SimplyTest is heading and can recognize what incentives come with their donations (aside from the service running).

There is a lot of strategy involved and many ideas I wish to explore in the coming and future years (e.g. infrastructure for a modernized system and a paid internship to help those underrepresented in tech). However, several people in the Drupal community and the Drupal Association, has joined as backers. I can’t thank you all enough. I welcome others to join as backers and will work to outline these opportunities and vision moving forward.

Conclusion

I’ll continue to invest my time to make SimplyTest the best it can be. It’s a labor of love but one of the most rewarding and unique contributions we have in our community. I couldn’t be more confident in where this is heading and look forward to many vibrant years ahead for the service.

admin Fri, 01/18/2019 - 14:56

Tags