When implementing a cloud-based talent management application, there will likely be data you want to migrate to the new system. Whether you are replacing an existing application or simply moving data from Excel spreadsheets over to the new application, you should have a data migration plan in place to ensure a clean and smooth transition.
Prepare for Migration
When preparing for your data migration, you will need to plan for these five items:
1. Identify core data elements
Having a clear understanding of which data is important to move into the new system is the first step in your data migration plan. Data mapping details are usually defined in a scope of work, but there are instances in which the details are not clearly spelled out. Making sure everyone has the same expectations is critical.
2. Identify supporting data elements
Now that we have identified the data elements we want to import, we need to determine if there are any supporting data elements required. For example, if you’re thinking about migrating Instructor-Led classes over to your Learning Management System (LMS), you will likely need to import all metadata associated with the relevant locations and instructors for these classes. Also, any fields that have a list of values associated with them will need to be re-populated in the new application.
3. Determine one-time vs. ongoing data feeds
Determine which elements need to be migrated just once versus those that need to be part of an ongoing data feed. For ongoing elements, determine the frequency and timing that needs to occur. Additionally, you will need to determine which files will be provided to you (i.e., full files or deltas (changes to the data since the last time the feed ran)).
4. Consider historical data
There are a few key areas to consider when you are bringing over historical data such as student transcripts:
- How far back in time should you pull the historical data?
- Should you bring everything or just objects with certain statuses?
- Are there any regulatory considerations?
Try to be as consistent as possible when determining which data is migrated vs. which is not migrated.
5. Map the data
Now that we know exactly which data we want to bring over, we need to determine how it will fit into our new system. This involves doing a field-by-field determination of where every data element will live and how to set other properties such as default values on the objects we are importing. Typically, a data element will need to feed multiple objects in the new application. For example, courses may touch upon classes, transcripts, rules, etc. Be sure to have a mapping of all objects that are required to support this effort.
Generate Your Files
Now that you’ve done the strategic work to prepare for migration, it is time to create your data files. Keep in mind these four items:
1. Auto-generated codes
Many applications provide the ability to auto-generate codes or IDs for records. While this is convenient for an admin creating new records, it often adds unnecessary complexity to the data migration due to needing to translate codes for downstream objects that reference a parent object. Avoid using auto-generated codes and use existing codes from previous systems when possible.
2. Unique Identifiers
Determine what is the best way to uniquely identify an object. This plays a role in determining if a record gets inserted or updated in the new system. Can a user’s username or email address change? If so, you don’t want it to be the unique identifier as it may generate errors or duplicate records in the future.
3. File formats, delimiters, commas in data, etc.
Decide how your file should be created, including:
- File extension
- Data delimiter
- Special characters
- Commas in data
- Date formats
- Preserving leading zeros
4. Testing Plan
First, validate the number of records imported match what was provided and address any errors that were written to log files. Next, plan to smoke test random sampling of records to make sure all data fields made it into new system. Often several iterations will be necessary to address all errors and data issues.
Cutover
When it is time to go-live with your new system, migrate as much data in advance of a blackout period as possible. There will be some data that will need to wait until your legacy system has been taken offline. The following are considerations for the cutover process:
1. Static Data
This is data that does not change on a day-to-day basis and can be loaded in advance of a blackout period. Often, you can communicate with your system admins to stop creating certain objects as of a certain date which will allow those objects to be migrated before blackout.
2. Dynamic data
This is typically data that will change daily, such as new transcripts or registrations getting created for users. This data should not be migrated until the blackout period of the old system to ensure all data is captured.
3. Timing
Be sure to give yourself enough time to migrate all objects. The blackout period should be long enough to prepare the final set of files and imports. If you have done sufficient testing, you should have confidence in the level of effort needed for cutover. This can range from several hours to a week or more.
Keep these best practices, tips, and processes in mind when implementing your next talent management system. Remember, a well-constructed data migration plan will result in a process that runs as efficiently and smoothly as possible.
