Speak with an Advisor
Our expert advisors can answer your questions and help fast track your VS Code road to modernization.
Shift-left test automation concepts for COBOL and z/OS applications
Term |
Description |
| Automated testing | Using testing tools and scripts to automate the execution of test cases, making the testing process more efficient and repeatable; a core DevOps concept. |
| Assertion | A statement or a piece of code that evaluates whether a particular condition or expectation about the behavior of the software is true, typically in the form of a boolean expression. Test4z includes an Assert API. |
| Black box testing | Focuses on the functional requirements without considering the internal structure or implementation details; based solely on external interfaces, inputs, and outputs, treating the software as a "black box". Test4z supports black box testing. |
| Code coverage | A metric used to measure the extent to which source code has been exercised by a test suite (as compared to the total lines, branches, statements, or conditions in the codebase); ensures critical functionalities and use cases are tested thoroughly, including edge cases and error conditions; one of the 6 Core Principles of unit testing. |
| Continuous testing (CT) | Closely aligned with Continuous Integration (CI), CT is an approach to software testing that emphasizes running tests at every stage of the SDLC to ensure changes to the codebase do not introduce defects and that the application remains in a releasable state. |
| Functional testing | Verifies the software functions as expected by testing its features against specified requirements; non-functional testing, by contrast, encompasses aspects other than specific behaviors of the system including performance, usability, reliability, and security testing. |
| Gray box testing | Combines elements of black box testing and white box testing; partial understanding enables testers to create more targeted and efficient tests while still maintaining a level of independence; used when white box testing is not feasible due to time or resource constraints. Test4z supports gray box testing. |
| Integration testing | A type of functional testing; verifies the interactions between different units, modules, or systems, ensuring they work together seamlessly; follows unit testing, where individual units (such as classes, functions, or modules) are tested in isolation; focuses on communication between components, data flow, error handling, and boundary conditions. |
| Isolation | Testing a component, module, or system in isolation from its dependencies; keeps the focus on functionality without being distracted by other behaviors; mocking is a common technique to achieve isolation. One of the 6 Core Principles of unit testing. |
| Manual testing |
Testers execute test cases by hand, interacting with the software application as an end user would, to identify defects, assess functionality, and ensure that the application meets specified requirements.
|
| Mocks | Objects that simulate the behavior of real objects in a controlled way to isolate the code being tested from its dependencies; ensures the focus remains on the specific functionality being tested; Test4z includes a Mock API. |
| Regression testing |
A type of functional testing that ensures changes to the codebase have not adversely affected existing functionalities by re-running previously executed test cases; can be conducted at various levels including unit, integration, system, and acceptance testing.
|
| Shift-left | In the context of testing, shifting-left implies bringing testing activities, such as unit testing, integration testing, and some aspects of system testing, earlier in the development process; before DevOps, testing was conducted after the development phase was completed, leading to potential delays in identifying and fixing issues. |
| Six Core Principles of Unit Testing | The gold standard, these principles serve as the basis for successful unit testing: 1) keep tests small, 2) run tests in isolation, 3) make maintenance easy, 4) swift and repeatable execution with a feedback loop, 5) keep tests independent, 6) align devs and managers on code coverage; Learn more |
| Spy | Test double or test object used to observe and verify the behavior of a component under test; used primarily in unit testing and integration testing to monitor interactions between the component being tested and its dependencies; aides in the creation of thorough and effective unit and integration tests; Test4z includes a Spy API. |
| System testing | Evaluates the entire system as a whole to ensure all components work together according to the specified requirements; encompasses functional, non-functional, regression, and user acceptance testing. |
| Test data management (TDM) | Creating, managing, and provisioning data for software testing purposes; aims to ensure that test environments have the necessary and realistic data required for testing scenarios without compromising sensitive or confidential information. |
| Test harness | A set of tools, libraries, and infrastructure that automate the testing of software components or systems; typically includes scripts or cases, data, drivers, mocks, and reporting/logging mechanisms. Test4z includes a Harness API. |
| Test4z | A component of the Broadcom Mainframe DevOps Suite, Test4z is a solution for unit testing and, more broadly, functional test automation. A code-based, CLI-enabled solution, Test4z equips mainframe developers to adopt shift-left practices using their tools of choice. APIs act as an abstraction layer to simplify test case authoring and a runtime executes tests and captures code coverage. Learn more |
| Testing frameworks | Also known as testing libraries; pre-defined sets of guidelines, rules, and reusable components that provide a structure for designing and implementing automated tests; often include tools, libraries, and conventions that simplify the process of writing, organizing, and executing tests. |
| Testing pyramid | Metaphor that illustrates both the granularity and volume of tests by type; unit tests are the foundation of the pyramid, integration tests are typically the middle layer and end-to-end tests are at the top. |
| Unhappy path | Testing scenarios where the system or application is subjected to inputs or conditions that are unexpected, abnormal, or erroneous; ensures the system handles errors, exceptions, and edge cases gracefully and appropriately. |
| Unit testing | A type of functional testing; tests individual units or components of a software application in isolation to ensure each part of the code works as intended (e.g., COBOL paragraphs); critical practice in software development that contributes to improved code quality, faster development cycles, and greater confidence in the reliability and maintainability of application. |
| Unit Test for Mainframe | Visual Studio Code extension that facilitates the use of the Test4z APIs and runtime by developers; developers can also use the CLI and/or connect directly using ISPF via SSH. Component of the Code4z developer experience. |
| White box testing | Also known as ‘clear box’ testing, this method involves the internal structure and implementation details of the software being tested by examining the code logic, data flows, control flows, and program paths. Test4z supports white box testing. |
Manual testing, which remains prevalent, is both time-consuming and risky. Conversely, test automation accelerates delivery velocity, resulting in shorter time-to-market for important product enhancements, while reducing the risk of code changes. It replaces the potential of human error with consistent and repeatable results. Automated testing also provides rapid feedback on code changes, allowing developers to catch and fix issues early in the development process increasing productivity and reducing costs (i.e., the shift-left value prop).
At a time when many veteran developers are retiring, it’s important to keep the next generation focused on coding with test automation in place. Let’s not burden another generation with manual testing.
Test coverage in the mainframe world is limited despite the long-standing availability of testing tools. These tools, built during the pre-DevOps era, tend to be GUI-driven which is not how developers want to work.
Developers love Test4z because it’s a code-based, CLI-enabled solution that easily integrates into their workflows, whether using ISPF, Eclipse, or VS Code, and into CI/CD pipelines for automated execution and reporting. Unlike the older tools, Test4z works just like test automation tools in the cloud/distributed world, ones your new-to-mainframe talent is already familiar with.
If you have experience with the legacy solutions and would like to learn about the Test4z approach, please reach out (see below).
Test4z is a self-contained solution, including native code coverage and record/replay as well as a build tool (Team Build), so no additional tooling is needed. Zowe is not required.
Our test automation experts are available to share adoption best practices to help maximize ROI. Automation requires an investment in developer cycles so it’s important to demonstrate the enterprise value of automation early on. Variables to consider include portfolio/application business risk, change frequency, and staffing/skills availability.
Copyright © 2005-2024 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries.
