Too Much Data: Collection System Data Consolidation and Visualization Using Open Source

Flow monitors are useful for both operations and data analysis, but the amount of data generated by them can be overwhelming and difficult to parse. In order to better leverage this data, we used open source tools such as Python and LaTeX to analyze the flow monitor data, create human readable and actionable data visualizations, and automate document creation. The final data product was a flow monitor report card. These report cards are now produced quarterly for several basins and have been useful in documenting performance of flow monitors across seasons and during varying weather conditions; they are also useful for providing a common language to discuss basin issues and possible solutions with Operations and Maintenance. We hope that this work will provide an example that other municipalities can use to evaluate their own needs and get the most out of the data they are collecting.