Skip to content

Data Management and Reporting

Home   >   Technologies   >   Data Management and Reporting

Traditional engineering approaches to data management have failed to keep pace with the exponential increase in the volumes and variety of data that we collect.  Even amongst large technically savvy companies with significant IT budgets, some analysts estimate that only 5-10% of all data collected is ever effectively analyzed.  So what chance small to medium-sized engineering companies with limited technical resources, and even more limited budgets?

 

The Two Traditional Engineering Approaches to Data Management

There are two traditional approaches to data management for small to medium engineering organizations, neither of which is optimum.

The first, and most common, is to store historical data as a loose collection of flat-files, and to analyze the data file-by-file using MS Excel, or a similar tool.  Storing data as a collection of flat-files distributed throughout the organization means there is no centralized data management.  Individuals often develop their own workflows based on standard Windows file functions, and it is difficult to enforce organization-wide standards for data security and data integrity.  It is also difficult to identify potential data correlations that may exist across multiple individual files.  Finally, MS Excel, whilst an excellent all-round business spreadsheet, is not especially optimized for performing engineering analysis, and it quickly becomes slow and cumbersome when working with large data files.

The second traditional approach is to store all critical engineering date in an industry-standard database such as MS Access, SQL Server or similar.  This addresses the issue of centralized data management, but at a significant budget cost, and often at the expense of flexibility in data analysis.  Databases are IT-centric tools.  They are expensive to set up, and require a significant commitment of IT resources to maintain.  The formal requirement to work across the IT interface can often be a barrier to the ad-hoc collaboration between engineers that is a critical part of the DNA of many small and medium-sized engineering organizations.  And as with MS Excel, databases are primarily business tools that are not especially optimized for performing engineering analysis.

financial candlesticks

In the right circumstances these traditional approaches can work, but for many small to medium sized engineering businesses the circumstances aren’t right.  If the traditional approaches aren’t quite working for you, there is a better way!

A Better Way to Manage Engineering Data

ICON Technologies can use the National Instruments Technical Data Management (TDM) Suite to implement a management and reporting system for your engineering measurement data that falls midway between the “flat files + MS Excel” approach and the “IT Department database” approach, and is specifically optimized for engineering analyses.

Structurally the data still exists as a collection of flat files in a local or network folder, and can be easily maintained by any reasonably computer-literate engineer.  But files that are dropped into the data repository folder are automatically indexed to support cross-file searches and analyses, even across different file formats.  Engineers can collaborate on ad-hoc analyses of data from a variety of different sources, all without needing to call on the support of a dedicated IT specialist.

You get all the integrity and security benefits of centralized data management, without the IT overheads associated with maintaining a full-on database.

The TDM Suite includes the DIAdem analysis and reporting tool that is specifically optimized for engineering type data analysis.  DIAdem can efficiently handle arbitrarily large data files, and is tightly integrated with the data management functions of the TDM Suite.  In our experience, most engineers that have a good working knowledge of MS Excel will not find it difficult to transition to DIAdem.

ICON Technologies can use the TDM Suite to create an engineering-centric data management, analysis & reporting system that is much more powerful than the “flat files + MS Excel” strategy, and simpler and cheaper to maintain than a full-on IT database.  Talk to us about how you can get more from your valuable engineering data.

A Deeper Look

The NI Website contains full details of the elements  that make up the NI Technical Data Management Suite.  The three major elements are:

  • DataFinder – Maintains the flat-file data repository.
  • Analysis Server – Allows user specified data cleanup and analysis routines to be triggered automatically when a file is dropped into the DataFinder repository.
  • DIAdem – An analysis and reporting tool that is optimised for engineering data.

ICON Technologies can mix and match these elements as required to create a data management, analysis, and reporting framework that integrates seamlessly with all your engineering measurement systems.

↑