Bulk load and integrate ILI, CIS and DA surveys to the enterprise database with powerful data alignment capabilities and flexible format support.
A common problem among pipeline operators is integration of survey data with the enterprise data management system. DataFrame™ Loader solves this by allowing users to easily load, align, and publish in-line inspection, close interval survey and direct assessment data to the enterprise database.
Typically, surveys use an independent referencing system, such as an odometer or interval, which does not correspond to pipeline stationing. Before survey data is integrated with the other datasets, it should be aligned to known pipeline locations to account for the difference between stationing and odometer readings, a process called Macro Alignment. DataFrame Loader examines the control and survey data to find the best alignment ensuring a valve from the survey is aligned to the known valve location with the same pipeline stationing. In essence, the alignment brings survey data into a single frame of reference with the enterprise data. All users of the data benefit from standardized formats and consistent storage locations, thereby improving the decision-making process. Users may also retrieve the original survey from the database if they wish to examine the details of the survey.
Survey and Assessment Loading
Surveys come from a variety of different vendors, and every vendor has a unique way of delivering the data. For this reason, DataFrame Loader is user-configurable allowing users to take any survey and define how the data is published to the enterprise database. The application gives users the ability to bulk load survey data from raw files, including GPS files, without needing a database administrator. Furthermore, DataFrame Loader verifies the quality of the data against the database and produces an error log if any issues are found. Once the data is published, defect characteristics are analyzed much more thoroughly because the anomalies are integrated with the other enterprise datasets.
Pipeline Feature Bulk Loading
DataFrame Loader offers the capabilities to bulk load external facility data to the enterprise database. The flexibility of the application reduces the time needed to format and cleanse the data before it is loaded. Quickly load an unlimited amount of data without the preprocessing and time historically required to get new data into the system.
Data Validation
Detailed data validation controls the quality of the data being loaded, helping ensure that imperfect data is not introduced to the database. As the data is being published, database code and stationing values are verified to ensure the published data is accurate and consistent. If an error is found, a detailed log is created to describe the error.
Multiple Data Source Types
The wide availability of storage software means the source data may be in different forms of databases. DataFrame Loader makes it easy to load data by allowing users to directly load from some of the most commonly used storage software. This includes not only Excel, but also Shapefile, File Geodatabase, Access database, ArcSDE, SQL Server database, MySQL, and Oracle database. Additionally, DataFrame Loader is capable of extracting coordinates from geometry-type files.
Data Mapping Interface
The data-mapping interface provides robust data mapping options to describe data translation and data conversion. These features greatly reduce the time needed to format and cleanse the pre-loaded data. These mapping options are saved and reused repeatedly.
Intelligent Data Alignment
Integrating in-line inspection, close interval survey and direct assessment data into the enterprise database is made simple with intelligent data alignment.
In addition to macro alignment, powerful new micro alignment capabilities of the updated feature matching interface allows alignment of new surveys to historical runs at a very precise weld-to-weld level. The user defines the alignment parameters for the control and survey data ataFrame Loader uses a DNA matching process to identify likely matches between the two datasets.
This automated and streamlined alignment process still allows the user to remain in complete control of the alignment with the ability to manually match and separate aligned pairs.