Source Code Maturity levels

Have you jumped into DevOps wagon already? You probably have. But perhaps you still not sure if you are lacking a certain tool in your toolbox if you are working currently with DevOps.

Or maybe your organization or team is starting to plan to fully embrace DevOps and your team is researching what is exactly what to need to install in order to have the perfect toolchain. Perhaps you have a gap in some processes that you are not even aware of. Establishing a good and solid DevOps toolchain will help determine ahead of time the grade of the success of your DevOps practices.

In this blog post, we will be exposing maturity level checklists for different DevOps areas so you have an idea where you at in terms of Continuous Delivery.

We will review the maturity levels from the following DevOps aspects:

  • Source code management
  • Build automation
  • Testing
  • Managing database changes
  • Release management
  • Orchestration
  • Deployment and provisioning
  • Governance, with insights

Source code management tool

Commonly known as repositories. It works as a version control and can be used to keep track of changes in any set of files. As a distributed revision control system it is aimed at speed, data integrity, and support for distributed, non-linear workflows.

This is the maturity level checklist. we go from a none or low maturity level to a high maturity state:

  • No version control
  • Basic version control
  • Source/library dependency management
  • Topic branches flow
  • Sprint/project to branch traceability

Source Code Maturity levels

Build automation tool

Continuous Integration (CI) is a software development practice that aims for a frequent integration of individual pieces of work. Commonly each person integrates at least once per day giving place to several integrations during the day. Each integration should be verified by an automated Build Verification Test (BVT). These automated tests can detect errors just in time so they can be fixed before they create more problems in the future. This helps to reduce a lot of integration issues since this practice allows to develop faster and in a more efficient way.
This is the automation maturity checklist to see how you are doing in your CI:

  • No build automation. Built by hand. Binary check-in.
  • Build automated by central system
  • Reusable build across apps/projects
  • Continuous/nightly builds
  • Feedback loop for builds

Automation Maturity Levels

Testing framework

Testing automatization can be in code, systems, service etc. This will allow the testing each modification made in order to guarantee a good QA. Even the daily or weekly release of code will produce a report that will be sent every early morning. To accomplish this you can install the Selenium app in Clarive.

This checklist will help to determine your testing practices level:

  • No tests
  • Manual tests
  • Automated unit/integration tests
  • Automated interface tests
  • Automated and/or coordinated acceptance tests
  • Test metrics, measurements, and insights
  • Continuous feedback loop and low test failure

Testing Maturity Levels

Database Change Management

It’s important to make sure database changes be taken into consideration when releasing to production. Otherwise, your release team will be working late at night trying to finish up a release with manual steps that are error-prone and nearly impossible to rollback.

Check what is your team’s database management current state:

  • Manual data/schema migrations
  • Automated un-versioned data/schema migrations
  • Versioned data/schema migrations
  • Rollback-enabled data/schema migrations

Database matutity levels

Since databases schema changes are sometimes delicate, make sure to include your DBA team into the peer review process, so that changes are 1) code; 2) can be merged and patched; 3) can be code reviewed.

Release Management and Orchestration

You can fully orchestrate tools that are involved in the process and manage your release milestones and stakeholders with Clarive.

Imagine that a developer makes a change in the code after this happens you need to promote the code to the integration environments, send notifications to your team members and run the testing plan.

Are you fully orchestrating your tools? Find out with this checklist:

  • Infrequent releases, releases need manual review and coordination
  • Releases are partially automated but require manual intervention
  • Frequent releases, with defined manual and automated orchestration and calendaring
  • Just-in-time or On-demand releases, every change is deployed to production

Orchestration Maturity Levels

Deployment tool

Deploying is the core of how you release your application changes.

How is your team deploying?:

  • Manual deployment
  • Deployment with scripts
  • Automated deployment server or tool
  • Automated deployment and rollback
  • Continuous deployment with canary, blue-green and feature-enabling technology

Deployment Maturity Levels


As part of deployment, you should also review your provisioning tasks and requirements. Remember that it’s important to provision the application infrastructure for all required environments, keep environment configuration in check and dispose of any intermediate environments in the process.

Yes, provision has also several maturity levels:

  • You provision environments by hand
  • Environment configuration with scripts as part of deployment
  • Provisioning of disposable environments with every deployment
  • Full provisioning, disposing and infrastructure configuration as part of deployment
  • Full tracking of environment-application dependencies and cost management

Provisioning Maturity Levels

We have come a long way doing this with IaC (Infrastructure as Code). Nowadays a lot can be accomplished with less pain using technologies such as containers and serverless, but you still need to coordinate all cloud (private and public) and related dependencies, such as container orchestrators.

In your path to provision automation and hands-free infrastructure, make sure you have a clear (and traceable) path to the Ops part of your DevOps team or organization, making sure to avoid bottlenecks when infrastructure just needs a magic touch of the hand. One way of accomplishing that is to have a separate stream or category of issues assigned to the DevOps teams in charge of infrastructure provisioning. We’ll cover that on a later blog post.

With the right reports, you’d be amazed by how many times releases get stuck in infrastructure provisioning hell…


Clarive has also productivity and management tools such as with Kaban swimlanes, planning, reports and dashboards that give managers tools to identify problems and teams a way to quickly check overall performance of the full end-to-end process.

Here are the key points to make sure you evolve the overall governance of your DevOps process:

  • There is no end-to-end relationship between request (why) and release (when, how, what)
  • Basic Dev-to-Ops traceability, with velocity and release feedback
  • Full traceability from request to deployment
  • Immediate feedback and triggers

Maturity Levels of Source Code Management

There you go, let’s devops like the grownups do

In this post, we have exposed the main Continuous Delivery aspects that every DevOps team should be looking forward to improve and their respective readiness levels. So go with your team and start planning a good DevOps adoption plan 😉

Schedule a demo with one of our specialists and start improving your devops practices.

Clarive Test

A test plan is an essential part of software development. It is a must if you want to get devs and ops people on the same page. As a guide and a workflow, it reinforces your projects success detecting potential flaws in advance. Tracking is also an important part of the testing process, as changes are applied, the test plan should be updated.

Unit test, code QA, performance test, integration and regression test are some of the most common types of software testing, and in some cases they are even compulsory to apply as a previous step to a production deployment.

With Clarive you can create a QA workflow that combines manual and automated steps, creating test plans automatically on a pull-request/merge-request, which in Clarive can actually be any changeset (user stories, features, bugfixes, etc.). Test plans can then be used to automate test validation.

Like in all software development processes, it’s necessary that each code revision go through a proper testing process to ensure the quality of the product.

Automating test plan creation

You can also create test plans automatically along with the included changes in the version to be released. This feature is capable of detecting the modified files by the developer, create a plan with the test cases (automated and manual tests) that have impact over the modified/created functions and link them to the release. Clarive will only add the test cases that are directly affected by one of the functionalities modified in the release.

This way Clarive creates a test plan suitable for each release and if the user wishes to, automate the test cases and depending on the results execute the required actions.

Download our whitepaper: The Value of DevOps for Test & Quality Managers and learn more about how to minimize the risk of product failure with Clarive.

Continuous integration (CI) is now becoming a standard in all software projects. With this, a pipeline is executed for each new feature or change in code that is made, and this allows you to carry out a series of steps to ensure that nothing has been “broken”.

One of the main steps that needs to be undertaken in the CI is the implementation of tests, both unit tests and regression tests. The latter can be done in the nightly build that Clarive is responsible for launching on a daily basis, at the time that we pre-define within our Releases.

Unit tests must be run whenever there has been any change in the code. This brings us to our first step. The unit tests must be run with each change in source code using a transfer (Continuous Integration) and the tests will be launched in the nightly builds of each new Release. This will ensure that the new version of our product will go onto the market without any bugs or problems in the features that worked well in the current versions.

At the end of this post the reader should know how to:
Integrate tests into a Clarive rulebook.
Publish files in an artifact repository.
– Send notifications to users.

Ready to start

To start this step-by-step guide you need a copy of Clarive installed on your server (you can request a 30-day trial).


For this post, we will use the following workflow:

1- User commit tests.
2- Clarive runs rulebook.
3- Mocha run tests.
4- Clarive post a message in Slack with the report.

For the first part of the development, we will assume that the developer has already written his tests. In this example we will use a simple “Hello world” and some example tests that you can find here and which will use the library expect.js, which will be installed during the running of the rulebook.

Workspace files:

In this example within our git repository we have the following file structure:

├── .clarive.yml
├── package.json
├── src
│   ├── add.js
│   ├── cli.js
│   ├── divide.js
│   ├── helloWorld.js
│   ├── multiply.js
│   ├── sort.js
│   └── subtract.js
└── test
    ├── helloWorld.tests.js
    ├── test-cli.tests.js
    └── test.tests.js

.clarive.yml: The rulebook file that Clarive runs
package.json: NPM configuration file. Within this file we have defined a command to run the tests that are found in the test folder:

"scripts": {
    "test": "mocha --reporter mochawesome --reporter-options reportDir=/clarive/,reportFilename=report,reportPageTitle=Results,inline=true"

src: Folder where we find the source code.
tests: Folder where the test files are located.

Writing a rulebook

The rulebook will be responsible for running the tests and generating a report to notify the user of the results of the tests.

First phase: Running the tests

As we can see in the file skeleton clarive.yml, there is a step named TEST that we will use to run the tests:

    - log:
        level: info
        msg: The app will be tested here

But first, let’s define some global variables:

  - workspace: "${project}/${repository}"
  - server:  https://<my-clarive>
  - art_path: ${server}/artifacts/repo/test-reports

We can see from the code that we are going to use node.js with mocha, chai and expectjs as libraries to run the tests. And thanks to Docker images, we can use a container with mocha and chai already installed, and this means that you only need to install expectjs using the rulebook.

We can now specify the Docker image we’re going to use during this TEST phase and then install the expectjs library from our working directory and, finally, update our package.json:

  image: mocha-chai-report
    - shell:
        cmd: |
             cd ${workspace}
             npm install expect.js
             npm install

Next, we just need to run the tests. As we have already defined the command to run the tests in our json package, we only need to run npm test:

  image: mocha-chai-report
    - shell:
        cmd: |
             cd ${workspace}
             npm install expect.js
             npm install
             npm test

Second phase: Publish report

When you finish running the tests, an HTML report will be generated with all the results, thanks to the mochawesome library installed in the Docker image. We will now publish this report in our artifact repository:

- publish_report = publish:
        repository: Test reports
        from: 'report.html'
        to: "test-reports/{{ ctx.job('name') }}/"
    - log:
        level: info
        msg: "Report generated successfully!"

Now, we are now going to complete the POST step. During this step we will inform the users that we want to complete the deployment.

Third phase: Notifications

We need to post the message in Slack, with a link to the report. To do this, we will have to have our WebHook configured in Slack and in Clarive:

Now all that remains is to add the sending of the message to the POST step:
From the Slack API any user can configure their own messages. In this case we have configured a message in which the user, on receiving the message, will be able to access the report from the same channel:

- slack_post:
          webhook: SlackIncomingWebhook-1
          text: ":loudspeaker: Job *{{ ctx.job('name') }}* finished."
          payload: {
              "attachments": [
                "text": "Choose an action to do",
                "color": "#3AA3E3",
                "attachment_type": "default",
                "actions": [
                        "name": "report",
                        "text": "View test report :bar_chart:",
                        "type": "button",
                        "style": "primary",
                        "value": "report",
                        "url": "${art_path}/test-reports/{{ ctx.job('name') }}/report.html"
                        "name": "open",
                        "text": "Open Monitor in Clarive",
                        "type": "button",
                        "value": "open",
                        "url": "${server}/r/job/monitor"

Lets make continuous integration!

Using Clarive, we can create a Story where we will attach all the code. If we haven’t already created a Project, we can create one using the Admin panel:

Then we create a repository of artifacts (a public one in this case):

Once we have carried out the previous steps we will create the Story in Clarive:

We’ll now change the state to “In Progress” and Clarive will automatically create a branch in the repository:

From the console we’ll clone the repository and change to the branch we’ve just created:

We’ll now place all our files in this branch and push them to the repository:

Clarive will automatically generate the deployment:

After finished, we will received the notification on the Slack channel that we’ve selected:

Finally, we can check test report in the artifact repository.

Next Steps:

In Clarive – After making this first contact with Clarive, our Story can be assigned to a Release (one that was previously created from the release menu) and, after changing the state to “Ready”, the developed feature will be automatically merged into the branch of therelease.

In Rulebook – Well, this is a simple example. The rulebook can be as complex as the user wants it to be, for example, by adding if statements, adding elements in the different deployments. For example, if something has gone wrong, create a new Issue in Clarive automatically, only run tests if the “tests/” folder has been modified in the commit…the possibilities are limitless!.

Visit our documentation to learn more about the features of Clarive.