For those who’ve participated in any system implementation or upgrade, you're well aware that data migration and conversion is critical to a project’s success and requires significant planning.
However, since migration is commonly one of the final activities before production, it often receives insufficient attention. Accordingly, this post will detail the best practices to structure and deploy a successful data conversion process.
In the ERP world, data migration takes into account the process of moving, copying, and reorganizing data from an existing system to a new one.
While data conversion is usually performed programmatically through source code, component interface tools, or pre-defined templates in the Cloud- manual setup and effort is still required to support master and transactional data migrations.
The following steps should be performed to execute a successful migration plan...
The first step is to identify which data needs to be migrated. For example, in a Financials ERP conversion, datasets in the Customer, BI, AR, AP, GL, OM, and Finance tables are essential for migration in order for the new system to perform seamlessly. In addition, previous customization data needs to be included, as well as the determination to whether or not include historical data.
We often find our customers asking us questions about the differences between Master and Transaction data, and their migration needs.
Master data are records that do not change often and are always needed in the same way by the business. For instance, one-time activities like creating company codes, chart of accounts, products, vendors, customers etc. On the other hand, Transaction data changes frequently and deals with the day-to-day activities carried out by the business. This data includes sales, purchases, deliveries, invoices, receivables, etc.
2. Extraction & Transformation
Once the initial raw dataset is agreed upon by the client and Emtec team, templates need to be built and populated in order to map the information from the old system to the new one. Dataset templates are frequently created and managed in Excel, .xml, and .txt documents. Additionally, Component Interface tools and custom templates are often deployed to assist with larger and more complex datasets.
To simplify the transformation process, Emtec has stored data extraction scripts from other on-premise Oracle ERPs (PeopleSoft and EBS).
At most companies, data has been stored on internal systems for several months, if not years. Consequently, data included in the extraction templates discussed above often contain mass amounts of information that is not standardized, is out dated or no longer used. At this step, it is frequently encouraged that the client spend substantial time reviewing, updating, and scrubbing old data to ensure that the new system is clean and up-to-date.
In almost all cases we urge clients to start working on data cleansing as soon as the project’s Analysis phase is complete. Unfortunately, all companies have data quality issues and the cleansing process takes a long time is usually allotted part time effort.
This step may appear obvious, but is nevertheless extremely important. Especially early on in an implementation, it’s vital that the conversion programs (manual or automated) are both unit and system tested with a subset of data.
Full dataset loads need to take place as the project moves from Development > Testing > Pre-Production > Production environments. Finally, the initial load phase is a crucial point to find and resolve errors in the overall loading process.
After loading data into each environment, the results need to be validated and verified to determine whether the data was accurately interpreted, is complete, and finally, whether it supports delivered processes and customizations in the new system. During the validation step, there may be a need for a parallel or regression run of both systems to identify areas of inconsistency and prevent data loss.
As an added benefit, Emtec has the necessary reports to allow for reconciliation in the new environment across multiple Oracle ERP platforms.
As the steps above detailed, data conversion is a central process to an overall project and requires both early-stage and detailed attention to ensure a successful rollout of a new or upgraded system. Please contact us for more information regarding our services around these capabilities.