This article is also available in PDF File.

"We have invested millions of dollars and years of time and energy into the creation of this system and I cannot get the information I want out of it!" Many SACWIS project managers and staff around the country have heard the preceding line, or one similar to it, as their projects have moved from development to implementation. SACWIS staff often feel that the users are making unreasonable demands on a very complex system. The sentiments expressed above may be the result of a range of different factors impacting on the reporting capabilities of a SACWIS system. This document will take a look at some of the fundamental realities around reporting in large data systems with an eye to bridging the disconnect between technical and program staff, avoiding disastrous consequences, and promoting successful SACWIS deployment.

Developments in Reporting
In recent years, reporting has undergone changes in the critical areas of purpose, tools, and user needs. Those changes are reflective of and at the same time agents of change in the thinking of system managers and users. If both parties do not properly appreciate the changes, disconnects like the one described above may result. Proper consideration of those issues is the key to developing effective solutions. The first issue to consider is the change in the overall purpose or nature of reporting. That purpose has changed from a fairly straightforward, descriptive effort to answer the question: "What are we doing?" The descriptive purpose still exists to meet basic Federal and fiscal reporting requirements. Today, however, reporting also encompasses an evaluative effort summed up by the question: "How well are we doing the things we do?" The evaluative perspective addresses more sophisticated performance and outcome measures. This development has changed the type of information users want in reports and introduced issues of policy and the input of program staff into the report design process. The second major element of reporting that has changed in recent years has been the development of tools. Reporting tools have evolved from complex mainframes to include an array of more user-friendly, distributed solutions. The development of new reporting tools has made it more cost effective to deliver some form of client-server reporting capability to individual end-users. Indeed, this development has accelerated in recent years with the introduction of web-enabled tool sets. All of this has had the effect of relieving IT staff from some of the burden of designing and producing all system reports. It also means, however, that choices regarding which tools will most effectively support an agency's reporting goals become more important than ever. A third and final key change in reporting has been a change in end-user needs. The days are over when users wanted only a very small amount of data delivered regularly in order to do their jobs. Today's users tend to be more comfortable with technology in general and with using data to do their jobs better. Many users no longer want to wait to get their monthly stack of mainframe printer output, they want real-time access to the data and the ability to perform customizable queries based on current needs. The change in user attitude and demands, coupled with changes in the overall purpose and tools available, has resulted in the need for a new way of thinking about reporting systems.

Issues in Reporting
While the issues covered in the previous section give a good sense of the changes taking place in reporting and some of the impact they will have on designing a reporting architecture, it leaves us with a few key questions: What to Report; When to Report; and How to Report? The following sections provide some of the lessons learned from Administration on Children and Families' User Group exchanges.

What to Report?
The primary lesson with regard to what to report is that end users should be involved in system design, review of existing reports, and planning for new reporting. When designing a new reporting structure, one should not assume that every report produced in a legacy system should be reproduced in the new system. User and program needs may have changed and some legacy reports may no longer be used or are duplicative of data presented in another format. Once that process is accomplished, it is important to begin to identify new reporting needs. User input must be weighed against the fact that needs are likely to change once the system is running and they can appreciate all the possibilities it holds. Start small and build from a solid base of critical reports.

When to Report?
The issue of questionable data quality as a barrier to reporting can become a slippery slope if system managers do not exercise caution. Data and statistical purists will note that the development of a new system opens many potential avenues for degradation of data quality. Certainly the conversion of legacy data into the new system holds a certain amount of risk. In addition, new users, many of them novice computer users, can enter bad data into the system despite the best of intentions. Some with a concern for the quality of the data may suggest that a new system not produce reports until data quality can be improved. Experience has shown, however, that staff may become disenchanted and less careful with regard to data quality if they receive no feedback from the system. The best results are often achieved if reporting begins with caveats about data quality-indeed those early reports can help suggest target areas for a quality improvement effort.

How to Report?
A discussion of how to report must begin with a disclaimer that there is no single best technology or tool available. Since SACWIS systems are complex data stores involving hundreds of tables and data elements, the ultimate "how" will vary from case to case. A reporting system is likely to be diverse with some reports delivered via traditional batch processes and monthly printer runs, while others may employ on-line delivery or access to customized data sets for ad hoc report development. We will be covering some of the issues around using specific types of reporting tools in future editions of "Tips/Tools/Trends." The next edition will be on data warehousing. In the end, the key to developing a successful reporting solution is to develop a good understanding of what is needed, design the reports with users involved every step of the way, and employ the right technologies to meet those needs.

This document was prepared as a supplement to the material presented in add-on sessions at the May 2000 ACF Users Group Meeting in Louisville, KY. It is the result of collaboration between the Office of State Systems and the National Resource Center for Child Welfare Data and Technology. The purpose of this document is to provide a brief overview of an issue relevant to the development or implementation of SACWIS systems and to the delivery of child welfare services. It is written with the goal of being accessible and informative to a wide audience including program and systems staff with varying levels of comfort with technology and policy. We hope that it will serve to stimulate an exchange of ideas and information among States and between systems and program staff. Your feedback is important to us. If you have any additional information on the topic presented in this sheet, or if you have any comments or suggestions regarding its presentation or content, please contact Tom Wetterhan of Xtria at (703) 821-3090 x250 or tomw@xtria.com.