Database Services for Software Engineering Departments

Introduction

Databases

Communication

They formalize ways for product team members to record, share, evaluate, improve, and implement ideas. The bugbase, in particular, becomes an efficient way for engineers to communicate about features and bugs.

Workflow management

Information about what work needs to be done, by whom, and in what priority can be bound directly to the details about that work. Ambiguity about what work needs to be done can be eliminated.

Status reporting

Project managers can quantitatively evaluate the current status of the project. Product readiness criteria can be fulfilled with confidence backed up by measurable quantities such as the number of open bugs and the rate at which new bugs are found.

Knowledge archive

When feature ideas, test procedures, and bugs and their fixes are recorded as part of the work process, valuable information and experience about the product are recorded in a central location. Employee turnover does not mean losing irreplaceable experience. Folklore is replaced by history.

Multi-user, multi-platform databases can provide a number of valuable services to a software engineering department.

• Feature Ideas
• Feature Tracking
• Hardware
• Test Script
• Bugbase
• Read Me Items

These databases provide specific services at every phase of software product development, throughout the product’s lifetime. That is, they are useful not only during the development of any given release, but also in reviewing past releases and planning future releases of the same and related products.

Feature Ideas Database

A repository and sounding board for ideas that can be developed into product features serves as a way to select features for successive versions of the product.

This database would be accessible by any member of the department and possibly any member of the company. Anyone in the company can add a new record or add comments to an existing record. This is the least formal of the databases, and company policy should encourage brainstorming here.

Field	Description
Submittor	Name of the person submitting the idea
Date	Date the idea was submitted
Source	Where the idea came from: magazine article, customer interview, dream, vision, hallucination
Classification	Must have, would be nice, pie-in-the-sky
Description	A text field for clearly describing the idea.
Comments	A text field for other team members to enter comments and suggestions about the idea.
‘Bang’	Evaluation of the usefulness or desirability of the feature to customers or users, made by the product management team
‘Buck’	Evaluation of the difficulty (cost in engeineer-days) of implementing the feature, made by the engineering team

When it comes time to determine the features of the next version of the product, each feature's Bang and Buck would be evaluated in light of market needs and development technology. The features can then be ranked by the bang/buck ratio and selected according to the time available in the development schedule, whether the feature fits in with current market needs, the current mood of the evaluator. When a feature is selected for development, it can be marked as “Implemented” (or even moved to an archival database) so it is ignored in subsequent product cycles.

Feature Tracking Database

Information on the current status of a product’s features and a summary of the current state of the product. The database would be accessible for reading by any member of the product team and for reading and writing by the engineering managers. The fields would be

Field	Description
Description	Text (taken from the corresponding Feature Ideas Database record)
Status	Where in the Development Cycle: Specification written, specification approved, test plan written, test plan approved, code in alpha, beta, frozen, testing completed, feature signed off.
Notes	Links to web pages containing the specifications, test plans, and so forth.

Hardware Database

Information about the configuration of each computer associated with the project. This can be valuable when isolating bugs that depend on the environment. The fields would be

• Owner
• Manufacturer
• Model
• CPU type
• CPU speed
• Physical RAM
• Number, type, and size of mass storage devices
• Type of video card(s)
• Type of network interface card
• Operating system configuration

Test Case Database

A repository of instructions for the test suite. Once the feature specification has been written, the QA Engineer will write a test plan. Once it is approved, the QAE will write a series of test script records that define the steps necessary to perform the tests described in the test plan. Any additional materials or files necessary to complete the test are stored on a file server. Keeping this repository will ensure that the tests are performed repeatably and correctly each time, even through changes in the QA staff.

Field	Description
Title	free-form text, one sentence to summarize the test
Number	automatically assigned, not modifiable
Date created
Date modified
Authors	who wrote the test
Project	selected form a standardized list of projects
Feature	selected form a standardized list of features
Subfeature	(optional) selected form a standardized list of subfeatures
Date run
Tester	who performed the test
Version	The version of the subject software it was last run on
Description	free-form text field containing simple step-by-step instructions to reproduce the bug and the expected results.
Results	Pass/Fail
Bugs	Links to relevant bugs in the bugbase.
Notes	free-form text field containing additional notes about this test.

Bug Database

A repository for errors found in the product, a tool for tracking those errors through the bug cycle, and a tool for measuring the current state of the development project. This is by far the largest database, and the one engineers would most interact with. It would have these fields

Field	Description
Title	free-form text, one sentence to summarize the bug
Number	automatically assigned, not modifiable
Project	selected form a standardized list of projects
Feature	selected form a standardized list of features
Subfeature	(optional): selected form a standardized list of subfeatures
Frequency	selected from {all users will encounter; some users will encounter; few users will encounter; not reproducible}
Severity	selected from {data loss, crash, blocking, failure, failure with workaround, usability, cosmetic/spelling/grammar, ECR, feature request}
Fix By	selected from {Next Build, Alpha, Beta, Final, Defer, Never}
Environment	operating system. This could be a link to a database of computer descriptions.
Reporting Engineer
Date
Version
Dev Engineer
Date
Version
Assertion	selected from {not asserted, assigned, as designed, not a bug, pending, checked in, fixed, need more info}
Verifying Engineer
Date
Version
Assertion	selected from {not asserted, agreed as designed, agreed not a bug, agreed fixed, info provided, not agreed}
Description	free-form text field containing simple step-by-step instructions to reproduce the bug, actual results, expected results, pertinent notes
Notes	free-form text field containing additional notes about this bug.

There would also be a place on a file server where any supporting files could be kept. If a particular file is needed to reproduce the bug, that file is stored in a directory with the bug’s number. To insure that the bug is properly regressed, that copy of the file must never be modified. A system for allowing beta testers to enter bugs through a web site is a useful way to engage their cooperation and help during the beta cycle.

Bug Reporting/Fix Cycle

The design of the bug database is closely tied to steps in the bug process.

Report

QA engineer (or beta tester) finds the bug, characterizes it, and enters it in the bug database.

Bug Review Board

QA manager and Dev manager review bugs to classify their severity and assign them to engineers for fixing or defer them to be fixed at a later date (or never). (Assigning specific people to scrub bugs may or may not be necessary depending on the number of bugs found.)

Fix

Dev Engineer fixes the bug.

Verification

QA Engineer performs the same sets that originally led to the bug; verifies that the bug has been fixed and that no additional bugs have been introduced.

Regression

On a near-final version of the project, QA Engineer performs all the same steps that led to severe bugs to certify that they aren't still happening.

The bugbase can then be used to create reports and metrics such as the number of bugs currently open, the find and fix rates, the find/fix ratio, the qualitative nature of bugs being discovered, and the current projected zero-bugs date.

Custom scripts allow bugbase users to filter the database for open bugs assigned to them. It thus becomes a convenient way for Dev and QA engineers to schedule their work.

Read-Me Database

A repository for items to add to the Read Me documents. It would be generated not long before a product release by copying descriptions of open bugs from the bug database. The project managers would review the items and for each write an appropriate entry for the readme. A simple report would then provide the meat of the readme document.

Database Products

Selection Criteria

Customizability

While our general bugbase needs may be very similar to those of 90% of other software companies, we will probably have some unique needs. The database product should allow us to easily implement custom features to address those needs. We may discover additional database needs not envisioned by custom product designers; a generalized database product that allows us to throw something together to meet that need is better than a specialized one that has never considered the possibility.

Interoperability

The databases should easily share information with each other. The database product should be able to export data to some format that can be read by its eventual replacement, should such a need arise.

Performance, integrity, availability

The database product should provide snappy response times, maintain the data in a usable format, and be usable any time

Multi-platform access

The database should be accessible on a wide variety of computer platforms so that users may access them on the computer of their choice.

Simple easy-to-learn interface

The closer the database can be to being a no-brainer to use, the more engineers can focus their valuable attention on the development task.

Freedom from unnecessary bells and whistles: a simple robust application that provides 90% of our needs is better than one with all the latest trade magazine checklist features that provides 150% of our needs but is flaky and brittle.

Industry standard

A database product for which it is easier to find knowledgeable power-users is better than something obscure no one has heard of. This will be an important consideration when the company grows and we may need a database specialist.

WWW-capable

It would be useful for beta testers to be able to enter bugs directly into a beta bugbase, rather than filtering them through QA engineers via email, telephone calls, and casual conversations.

Product Options