Evidence – A powerful talk on Software Testing

I am quite lucky that I get to attend a few seminars and workshops where I come across some great quality content. I recently attended the Agile Testing & Test Automation Conference organized by knowledgehut in Singapore, and one session stood apart from the rest.

Very tastefully titled, the session was about the lacking proof of what was really tested when we say a build is QA tested and ready for production deployment. Title of the talk was “Evidence – What really did get tested and can one Prove it?” Speaker was Tim Carpenter, Director – Head, Test Engineering, at Standard Chartered Bank, Singapore.

We all run projects where QA is an integral part of software development. In fact, in the Agile / Scrum model, testing is hard-coupled with the development, so much so that many companies follow TDD (Test Driven Development) as the formal approach for the code development itself. Still, there was something radical in Tim’s talk.

He was doing something different, he was questioning the fundamental transaction of code drop from a developer to a tester. This gravitates to integrity if you take it personally but Tim’s take was rooted more on factual issues with today’s testing practices prevalent in the industry.

Most of the cases we just believe when a story is moved by a Developer to a QA that unit tests were done and now it is QA’s responsibility to proceed. What Tim mentioned, and I agree to it is that in DevOps and especially in Agile DevOps model, the Regression should be done by developers themselves with Unit tests and ensure they didn’t break anything. We’re primarily shifting the onus/responsibility to Developer to confirm that he didn’t break anything in the system.

Credits – The actual slide presented by Tim during the Talk. 

To have a system driven authenticated/validated evidence, Tim introduced a concept of the Release candidate, that is the current version of the system on which you’ll be adding your feature, code or process addition.

The normal expectation is, that the developer will run his / her tests on the release candidate (which is version controlled) and submit the evidence as the test run reports against the Version# of this Release candidate.

What all can be part of this?

  1. The business logic that is getting changed or added
  2. Any new batch jobs that are getting added to the scheduler or any existing ones if changed in frequency or duration
  3. Any configuration changes that are coming to effect
  4. Deployment scripts (if changing, else default version of the scripts will be recorded)
  5. Any new VM or infra provisioning that was done under this planned release
  6. All Internal tool/product binaries
  7. If this is an external/third-party tool, then the binary version of those too (to track on what version/patch this release was developed & deployed)

When the final release is made after a successful QA & UAT, then all of this and additional artifacts like Actual code (versioned), Build Process, & CI/CD Pipeline (code/version reference) should be recorded in the Code & Binary repositories.

By all standards, this is a very clean management of the build (including pieces like code, test cases, Binaries). Only if we could manage to achieve this in the real world. Adoption of this will solve the problems for sure, but we need courageous Project managers and team awareness to follow this to the T.

644total visits,1visits today