Everything you always wanted to know about DevOps shortly described: origins, principles, use of methods and implementation DevOps in Redmine. .
The origins of the DevOps movement in the IT world are dated around 2009. Shortly speaking, it is a methodology where development (Development, Dev) is tightly integrated with the process of using the product (Operations, Ops). DevOps is changing permanently, and it gives maximum flexibility because the product is updated over and over again up to hundreds of times a day. Then developers are monitoring and analyzing reactions of the users, and after that, they write new requirement specifications for development. It is the process of continuous development and deployment.
The origins of the DevOps
First, there was regular or « cascading» (waterfall) software development. It looked as follows: the customer brings the requirements to developers, they take them, go and "create" the products for a couple of months and present the final product. Everybody believed that it was a right approach for work. Why do we need to invent anything else?
The hatchet was buried in the changes made to the expected final product. In theory, a "typical" product can be designed quickly. You just need to make an A4 sketch with some notes and proceed to the Development department without any additional questions upon the processes. In the received outcome, you do not need to make any changes to the product or perhaps just polish it a little bit here and there. If you need to update, change, or add something, it can be done quickly and without too much additional cost to the basic estimation.
However, in reality, most of the projects are complex. Often the customer or ordering party is not 100% sure that everything how everything should be done and what is the expected outcome. The analysts do their best to ask the right questions and understand the answers, but the chances are high that at the end of the day no one knows how the final product should look like and if it will be accepted at the end of the project. You can work on prototypes, mood boards, and sketches as long as you want, but only users can tell you how the product should work.
Then came a new stage in software development, a time for flexible Agile project management methodologies and approach.
In Agile all the processes are based on the principle of close and constant interaction between customers and developers. The sequence of actions: a client brings the list of requirements to developers; they break them into steps/tasks and organize them according to the priority for the final version of the product. Then the assignments are scheduled for one or two weeks pieces called Sprints and executed. Then the developers close, perform and coordinate with the customer a test version of the product, delivered withing the scope of on Sprint. When it is ready, they launch a new product to the market, check it, and correct errors. And then push the process on the new circle over and over repeatedly, until all the objectives are not met, and the product is functioning optimally.
A key feature of Scrum is the ability to run immediately. However, customers usually don't use this advantage. Why? Let's talk about it the other time.
I do not see any problems if it is necessary to make changes during the work process. There is a possibility to add new functions to the next round of tasks, and the team performs them.
If you see that the functionality of the test product receives negative feedback from users, it means it is time for reconsidering it and making changes.
So here are the visible features of the new approach:
- The ability to manage changes and implement them immediately
- Capacity to test the project on real users rather than to simulate the situation.
Everything is fine in this approach except a significant problem without the solution. The barrier between clients and designers is finished, but the wall between developers and IT-department (testers and admins who are responsible for bring the changes to live) is still there.
Again, we are entering a new stage. Welcome, DevOps!
What problem are solved by the DevOps?
It is clear that functionalities are developed in "packages" even if the team is working on the Scrum. Then as "packages" are tested and finally deployed to a working product. So the product is developed in jumps only. When it is ready, we realize that Agile development becomes almost an analogy of waterfall methodology.
For a product that has already been launched to the market, there is a risk that if users do not approve and use one of the features from the Sprint, you shall change all the related functions. Everyone knows that it would be great to introduce a single functionality, check the reaction of users and immediately decide what to do with all associated features. This way, you can achieve almost 100% of flexibility without unplanned extra work, right? DevOps is helping to solve exactly the problem of unexpected changes. However, besides this, it resolves another problem of many IT companies. The barrier between developers and "IT people."
Let's take a look at reasons for the hidden conflicts:
- Developers prefer updates and changes. They get paid for it. IT department likes stable situations and prefers a stable product. It brings less risk that something breaks.
- Developers work "strictly by the instructions», IT is trying to do the best for users.
- Developers work in their environment, while the "IT guys" personalize the environment for the testing purposes.
- Developers often do not have enough IT knowledge, and IT specialists, on the contrary, do not know how to code.
- Developers and IT do not communicate with each other often enough. They are not aware of the issues and each other's needs.
As you can see, actually the problem has two sides. The first one is the technology and processes. The second side is the human relations. The synthesis of these factors creates a particular mood within the team. Often you can see that developers and testers/admins are the two fighting sides, what slow down all the product development.
How does the DevOps help to solve the problem? With the three fundamental principles:
- According to the methodology, it estimates the performance of the whole system rather than specific sections described by departments, developers or administrators.
An essential task for a manager who implements DevOps is to bring all participants to work as one team, trying to achieve the primary goal: to increase product value for the end users.
- Simple feedback system "right-to-left"
It indicates that the product is always evolving, the users' reaction is monitored. As new challenges are discovered, they are followed by creating tasks and immediately pushed for development. This "pipeline" implies an appropriate level of process automation, test and deploy, and leads to an open and frequent communication between the employees.
The goal is to create a culture within the team which stimulates experimentation. We cannot always understand the level of the product functionality which works as efficient as possible without testing on its final customers. That is why it is important to ensure that each member of the team is aware that he/she can bring some valuable insight and take an active part in analyzing and designing its future development stages.
Does the DevOps methodology work?
The principal question that always confronts developers who implement DevOps: do the customer need such process?
All depends on the size and complexity of your product or projects. Also, on the process of allocating resources. As a matter of fact, to be able to track all the processes associated with the project, you would have to deal only with it and nothing else. To improve the work efficiency and fasten the delivery flow of new functionalities there is a wide selection of methods. They all have different difficulties to be implemented, so the real question here is: is it better and more worthy to work with or without them.
It is reasonable to apply some basic changes but not to move all the mountain. In other words, we do not need to bring the process to the ultimate perfection - we can just start with few improvements to our processes.
DevOps is a great approach works for big projects, like developing popular web services (it is used by Dropbox or Flickr), or complex IT projects systems, like banking software. When we look at a continuous flow of improvements and fixes, we can assign separate resources for ongoing analysis, and experiments. It helps to plan ad allocate their resources for improving the product in advance. If your project is not that big, you can try to start with the implementation of flexible Agile technologies.
DevOps in Redmine
To describe the process of implementation DevOps in Redmine, let’s divide it into two elements: tools and methods.
First, would cover the tools we need to be able to coordinate our projects according to DevOps principles smoothly. To start, we need a CIS (Continues Integration Server) for checking all commits (changes) in the developed software. It helps to manage development and deployment process. So the best way to start would be with Jenkins Plugin. Jenkins in an open source automation server written in Java. It helps to integrate both testing and deployment procedures.
The core advantage of Redmine is that it helps to manage team communication, project management, along with the history of all the of changes, comments, code commits and all the other details necessary for a clear overview of development and deployment processes.
Integrated code commits are connected with tasks and project. It means that all team members can see the big picture of current spring, as well as the overview of the final release. It includes information which changes are done, to which modules of the software, how they affect the other modules and all how it is integrated with remaining ones.
Then the commits are sent to the repository. It ist related to issues, and you can see commit statues integrated with issues. That means you can quickly see what's done, in what's the current status of each part of the software that is being delivered. It gives good understanding, for example, issues with status Done are already finished in the development cycle and can be passed further. Adding status "Solved" means issues that also pass the stage of testing successfully and can be put to release by the IT team.
The second mentioned element - methods - describes the organization of work and processes inside the Redmine. To fully control the DevOps, manager needs to be able to see all tasks that should be developed, and the one that is already delivered and needs to be tested on real users. For example, we solved this in our company by using two separated dedicated Agile boards: one for development and one for deployment. Each board is divided into stages appropriate for each part of the project, so Dev Board covers new tasks, the one in progress and finished, and the ones that are ready for testing. Deploy cover issues that are already available for testing (with a division for sub-stages) and documentation (also with division). Additionally, you need to take care of constant communication between two camps and the constantly pushing for progress and improving the product.
In our example, DevOps helps in delivering many changes in shorter time and with more proven changes that closer corresponds with plans and initial needs.
For further reading
To read more about the DevOps, I recommend lecture of The Phoenix project: a novel about it, DevOps, and helping your business win written by Gene Kim, Kevin Behr, and George Spafford.
Has anyone of you, dear readers, implemented DevOps in your teams and can share the experience? I would love to hear your stories and opinions.