From Zero to Zen: With Xray Documentation

The Early Days: Navigating with Google Sheets

As a long time veteran at Mixmax, I witnessed the challenges of scaling up our test case management. Initially, we relied on Google Sheets. It was a straightforward choice for a team already using the Google Docs ecosystem: lightweight, accessible, and incredibly easy for anyone on the team to create and share test cases.

But, as we added more engineering teams and dealt with increasingly complex test scenarios, the cracks began to show. Our Google Sheets became scattered and unwieldy, making it difficult to locate existing test cases. The lack of standardized formats led to inconsistent information capture, creating gaps in our testing.

Transitioning to Notion

Seeking a better solution, we turned to Notion. Our Engineering, Product, and Design (EPD) teams were already using it for project planning, so it seemed like a natural progression. Notion’s databases promised better organization and sorting capabilities.

However, we soon realized that maintaining and configuring Notion pages and databases was more complicated than anticipated. Despite being a step up in organization, the platform still fell short in standardization and ease of finding information. It was an improvement, but not the solution we were looking for.

Discovering Xray for Jira

Then, we discovered Xray for Jira. Since our EPD teams were already utilizing Jira for task management, integrating our test cases into the same environment felt like a no-brainer. Xray offered an excellent interface for creating and organizing test cases directly within Jira, aligning perfectly with our existing workflows.

We created a dedicated testing board for each engineering team and organized test cases into folders for each product area. This structure made it incredibly easy for team members to find relevant test cases, significantly improving our efficiency.

Test Cases as Documentation

Another significant benefit was how our test cases doubled as product documentation. Our testing boards in Xray became reference points, clarifying how the product should behave in specific scenarios. This was especially useful for new team members who may be less familiar with how the product should work.

Integrating Automated Test Cases

In addition to manual test cases, we also utilized these boards to organize our automated test cases for each product area.  Each automated test case was documented with descriptions of what the test covered. This transparency made it easy for anyone in the company to see which test cases were automated and understand the specific functions these tests were designed to perform.

By having both manual and automated test cases in one place, it became effortless for our EPD teams to understand the scope of our test suite.

Harnessing the Power of Xray Test Executions

The benefits of Xray don’t stop at test casts either. Xray Test Executions revolutionized how we conduct and track test runs. With its visual interface added in Jira, developers gained instant insights into the testing status of their projects, streamlining communication and enhancing transparency. Treating test cases and executions as regular Jira tickets allowed us to link testing directly to developers' tasks, keeping everything organized and interconnected.

Looking Back, Moving Forward

While we haven’t yet implemented every feature available to us in Xray, reflecting on our journey from Google Sheets to Xray, the evolution is clear. We've not only streamlined our test case management but also enhanced team collaboration and tracking efficiency. Xray has been instrumental in adapting to our growing needs, proving to be an invaluable tool in our QA toolbelt. We are excited to continue to explore all of the features Xray brings to the table.