By Pete Cheslock, Senior Director, Operations and Support at Threat Stack
Last week, I had a call with Gene Kim, founding CTO of Tripwire and author of The Phoenix Project (see end of post for more details). I've known Gene from the DevOps community for awhile now, so we took this time to dive into all things DevOps and Security, in the end resulting in this great Q&A to share with you all on what bringing Security into DevOps means for us all.
Gene kicked off our discussion with a few questions for me:
How in the world did a nice DevOps person like you end up in the bowels of Infosec? Usually it works the other way around -- the smart Infosec people flee to saner grounds like DevOps.
While looking for my next opportunity, I wasn’t looking specifically looking for a job in the Infosec field, or even at security specific companies, but after getting introduced to the leadership at Threat Stack, it definitely opened my eyes to a whole new world that I felt like I was missing out on. After spending more time learning about the product and the marketplace, what I saw was actually a convergence of Infosec and DevOps much like we saw when Dev and Ops teams needed to fundamentally change their thought process in order to win.
What attracted me even more to this space, and the visibility that our product could facilitate, was that we could provide this service to companies that likely didn’t have a dedicated Security team to review and monitor system usage and access. As we see more and more companies of all sizes undertake these cloud initiatives, deploying net-new projects into places like Amazon, Google and Azure, Infosec teams become new barriers to progress, in ways similar to the battles between Development and Operations. I see a world where we can provide deep insight into services, users, and activities that these companies need, and provide this information to Devs, Ops and Infosec users of all kinds. Furthermore, we can embed this visibility and monitoring into the workflow, allowing companies to deploy more scalable and elastic infrastructure.
As companies move towards microservices and containerized deploys, it will be even more critical that the business is continually monitoring and analyzing the scope of changes to their systems. These monitors can (and should) be integrated early in the development pipeline, as Developers and Operations engineers build these complex, distributed applications.
Here’s a quote from my good buddy Josh Corman:
“If there’s one message that everyone in Infosec should know about the DevOps community, it’s this: DevOps is waiting for Infosec with open arms. Come on in, the water is awesome.”
Do you agree with his thesis?
It’s been an exciting time watching as DevOps and the overall community around that movement has matured over the past 5 years. We’ve seen more and more companies of all sizes make amazing organizational changes and fundamentally shift how they do business online. That being said, many engineers and technical leaders were resistant to that change, fearing “yet another buzzword.” But these DevOps concepts have now been around long enough for people to see how companies have been able to complete these large transformations. We’ve seen how to get teams with competing interests a set of shared goals and ideals in order to change how they get work done.
I see the same thing when it comes to the Infosec teams and security minded folks within companies. At many of these companies, though, the Security teams don’t have a seat at the table. They are getting shot down while the rest of the organization is making changes at an incredible rate.
So how can we enable Security and Infosec teams to embrace this new world of continuous deployment and elastic infrastructure? Much like we saw for the DevOps world, it will come down to a mixture of culture change and improved technical applications that will facilitate the integration of Infosec into DevOps. Much like how Chef and Puppet enabled teams to more effectively build and deliver highly scalable systems, I see companies like Threat Stack poised to deliver the tools to allow deep insight and visibility into the applications and services being deployed.
I then had some questions for Gene:
It looks like enterprises like GE Capital, Macy’s, Target, and Nordstrom are early adopters of DevOps in the enterprise; how does Infosec need to change when more of the Dev to Ops value stream migrates to DevOps patterns?
This is one of the most exciting things about hosting the DevOps Enterprise Summit. We have over 50 leaders from large and complex organizations, just like you’ve mentioned, presenting about how they’ve transformed, and how Development, Operations and Infosec have worked together to replicate the amazing outcomes that we typically associate with the DevOps unicorns (e.g., Etsy, Google, Amazon, Netflix, etc.).
Of course, in reality, the unicorns are multi-billion dollar organizations and complex organizations in their own right. However, when you’re an enterprise horse, you have to deal with very powerful and entrenched silos in Dev, Test and of course, Infseoc.
My belief is that we’re going to see the Infosec function transform just like QA/Test is transforming. In other words, in high performing DevOps organizations, you very rarely see a QA department that is writing and running the tests. Instead, QA is helping coach Dev on how to write good test cases and ensures that the right feedback loops exist so that Dev can validate that they’re achieving the functional and non-functional requirements (like Infosec, for example).
In this world, Infosec is not doing the security scans, nor is it pestering Dev and Ops to look at their reports. Instead, they are helping create the automated tools so that Dev and Ops can get fast and constant feedback on whether the code and environment are achieving the security objectives.
My favorite example of this is the three-year transformation of the Twitter Infosec function, which started when the @BarackObama account was hacked, resulting in a FTC injunction, requiring that Twitter be secure for the next 15 years. It’s an incredible story of how they integrated Infosec into the daily work of Dev and Ops, with the primary mission of not getting in their way.
So how are fast-growing companies implementing the DevOps principles of ownership and accountability while requirements for access tighten (SOC2/FISMA/PCI, etc)?
It’s often said that the main obstacle for DevOps adoption in large enterprises is Infosec and Compliance, and you can hardly blame them. For decades, both Dev and Ops seem to have done everything they could to fix security defects that are exposed late in the project lifecycle.
But what every Infosec and Compliance practitioner needs to know is that DevOps is the best thing in at least 20 years to happen to our field.
When Dev and Ops embrace DevOps principles, we fully embrace all the non-functional requirements, like performance, quality, reliability, and yes, security.
We want to know when we’re writing or operating code or environments that aren’t secure.
Because DevOps organizations are constantly doing deployments, and those deployments take minutes or hours, the “find to fix” cycle time is very short.
So the days of Dev or Ops taking nine months to get an urgent change into production, (or maybe a week if we break all the rules, often creating massive chaos and disruption), are coming to end.
DevOps value streams that sustain tens, hundreds or even thousands of deployments per day (such as at Netflix, Etsy, Google, etc), can’t be done without a ton of effective controls.
In fact, if you count the number of controls that are working in their deployment pipelines (e.g., automated test suites, security scans, performance testing, manual peer review of changes, deployment validation, performance testing, etc.), you’ll find FAR MORE controls in a DevOps organization than in a traditional waterfall SDLC.
Wrapping Things Up
Gene Kim is hosting the DevOps Enterprise Summit in San Francisco from October 21st to 23rd. Use promo code “THREATSTACK20” for a 20% discount -- Expires 10/10!
Coming up in just one month, Threat Stack will be hosting Gene Kim at our booth during the AWS re:Invent Conference. Stop by booth #742 on Wednesday, November 12 from 11am-12:30pm to meet Gene and get your free signed copy of The Phoenix Project.
We look forward to seeing you at re:Invent!