On why having a DevOps q&a website would be useful for many

Question

Proposal: DevOps

Unfortunately there is currently too much (negative) hype attached to the word devops. But there is also increasing interest into this field, as you can see new proposal are being opened on topics that would be a subset of this one (namely, continuous integration, chef, software quality assurance...). So I am trying to reboot this proposal, which already had many good questions here: http://area51.stackexchange.com/proposals/66151/devops

Trying to stay away from that (the hype), and simultaneously trying not to start the customary religion war between SOers doing sysadmin and SFers writing code, I can tell you what are the responsibilities of my work position, and why my questions would most possibly be off-topic on stackoverflow and serverfault (and several questions that are already in the proposed examples were real questions which had been closed). Sometimes because they are just off-topic, sometimes because opinion-based, sometimes just nobody will answer them. Note please that all of these examples do not define a different scope to the questions, but do address and require a different scope for the answers: so that a "give me options for task" question that would rejected on SO and SF would be well accepted here (if it is not blatantly lame, obvious).

development support

This spans from configuration of a code-review system, like gerrit, or static code analysis, and how to let these components interact. In part it's about procedures and guidelines, in other parts it is scripting, it often requires programming, and always basic sys-admin.

• how to require at least three +1 CR votes in gerrit for approving a changeset?
• how to let sonar cast votes against CR instead of V when integration test coverage is below a threshold?
• what is a git repository setup that allows snapshots releases of components in the dev environment, and two differently versioned release branches for staging and production?

continuous integration

This boils down to integration testing configuration and reporting, and homogeneity of components build processes (well; a lot more actually, but I don't want to start my daily rant against agile here); would mostly be a sysadmin role against a build server, but requires very deep knowledge of build tools, the language, how to write good tests, and testing code in general.

• how to classify maven-generated rpm packages of spring java components in nexus, for the different deployment environments?
• functional/integration test failing with unit tests successful? how to report/what to do?
• how to enforce semantic versioning in commits, accordingly to integration tests results?

continuous deployment

This is power-scripting helped/depending on very different, but very specific, tools (chef/puppet/rundeck/...). While it is true that the provisioning of standard services to service nodes is pure sys-admin (i mean tomcat/postgres/postfix/....); the same for in-house components composing a platform requires very strong understanding of a software architecture.

• what is a good way to mark a nightly compose as a "good nightly", so that the most recent is used as a staging release-candidate when the sprint ends?
• should deployment of development environment be triggered by the code review code repository, and staging/production from the canonical one, or every envronment from different branches in the canonical one? (note: this is primarily opinion-based on purpose)
• how to setup the chef environment so that the httpd proxy load-balancer is updated when a new node of an horizontally scaling component is deployed?

monitoring

This is not monitoring of liveliness of a smtp server on a random linux distribution; it is the glue between monitoring facilities exposed by the deliverables and external monitoring tools. This need strong understanding of the software architecture, development skills for this specific duty, domain knowledge. Counter-intuitively, this would be most often closer to being on topic on programmers, then server fault.

• how to write the xml configuration of an hyperic agent plugin for the mbean exposed by spring in this way?
• what mechanism is available for dynamically changing the threading of an activemq queue consumer accordingly to the fact one broker of a two nodes failover setup failed?
• how to write a EL query that returns this, given that setup of logstash tags?

In doing my daily job of build engineer, I ask myself hundreds of these questions. Sometimes stackoverflow fits, sometime server fault does (less often); anyway, it would be awesome to have a tool of the same utility and helpfulness of stackoverflow, to answer all the questions that would be marked off-topic, getting help and letting share my knowledge with others.

Answer 1

The emergence of cloud or cloud-alike technologies modified the landscape of software development and deployment and brought new perspectives and challenges. Because we have the possibility to work with ephemeral servers, we want to take advantages of it. Instead of upgrading third-party software, we test and deploy on brand new, up-to-date, servers and ditch the old ones. Instead of carefully removing rootkits from compromised servers, we replace these servers, taking the compromised ones out of production for careful analysis, before ditching them. Instead of working with huge powerful do everything servers, we rely on dozens or myriads of smaller servers.

We need technical solutions

• that work well with ephemeral resources.
• that scale properly to handle an arbitrary amount of resources, we need to continuously adapt resources capacity to resources usage.
• that scale properly to be applied at high frequencies, deployments can occur several times a week or a day.
• that efficiently abstract resources to let us focus on services. (Efficiently does not mean totally!)

This perspective shift justifies the emergence of devops as a new discipline and e need to build a pertinent and easily accessible technical literature corpus considering problems from this very perspective. The Q & A format on SE seems appropriate, but this perpective shift also implies a shift in how we evaluate the relevance of answers to technical questions, and it does not seem that a SE-site where all devops themes fit already exists: these are two excellent reasons to found a dedicated community for us to share our knowledge and experience.

Answer 2

Given questions like this -> https://meta.stackoverflow.com/questions/294923/are-ansible-puppet-chef-salt-questions-on-topic on meta stackoverflow, I think it's pretty apparent that this is needed.

Switching from being a front-end Delphi programmer to a total back-end Linux guy, I'm seeing diminishing returns on crafting good questions on stackoverflow and even less on programmer stackexchange, where everything is off topic.

Unix and Linux is probably currently the best place to ask questions about most devops stuff, but I don't see why this site wouldn't include some of the fancier powershell/azure stuff that is permeating the ecosystem (at least the corporate ecosystem).

Answer 3

Here is my rationale for saying "yes".

1. Solutions architecture is what I do. Devops is how I do it. I apply Devops methodologies in at least three areas: 1. Software engineering; 2. Operations e.g. systems engineering; 3, Security.

Currently, something like 70% of Devops individuals define Devops in terms of their work in software engineering, 30 % define it exclusively in systems engineering terms as Systems Reliability Engineers, and 0% or close to 0% don't define it at all in terms of their security work. In other words, the various definitions of Devops that have been bandied about are incomplete. And those practitioners of Devops are guilty of more than a little tunnel vision when they have been peddling their definition of Devops exclusively in terms of the limited scope of the work e.g. exclusively software engineering, etc. - that they do. And when they do that, they create no small amount of confusion among everybody else. This reminds me of a bunch of blind men touching the various parts of an elephant, and describing the elephant as anything but an elephant. I am very frustrated with these people and their act: It's time to describe the elephant as an elephant.

One more time: Devops is a methodology, and the methodology makes sense and is applied in specific contexts such as software engineering, systems engineering and security. Absent these contexts, Devops doesn't mean a damn thing.

We need a Devops site that defines Devops and the contexts of Devops accurately, and that has its act together.

2. Currently, I can't think of a single site on SO where:

I can ask about how to build continuous integration pipelines.

Integrating Docker into continuous integration pipelines

Interfacing the components of a continuous integration pipeline

Discuss the pros and cons of various orchestration tools

Cloud operations

The different types of automated testing a Devops should know.

etc.

I am looking to build Big Data pipelines - that's probably part of my Devops obligation, too. As in, whatever it takes for a more efficient, more reliable workflow.

If you are a Devops and you never ran into workflow issues that didn't drive you crazy, then you aren't a Devops.

Answer 4

You may be surprised but I don't really see a need for DevOps, I got very good help on SO for all my DevOps related questions, and at the same time very bad feedback from SF site, so you can imagine what I am currently using.

I don't know if my good experience is because of my high SO score, but the feedback was great, even if in many cases it was borderline.

Use you imagination to make the questions not sound subjective, and SO will give you the answers.