[Written by Benj Fredrick for DTEK]
Upgrading a complex Drupal site can be a tricky proposition. This post will explore using the Migrate module to bring users and their role assignments from a D6 site to a fresh new D7 site. We created similar migration classes for taxonomy terms, nodes, and comments, which will not be covered (see the ‘Resources’ section below, or leave a comment if you need some direction). This is a followup to my last post about upgrading from Drupal 6 to Drupal 7, and presents a potential alternative approach. Before the migration, we built out the new Drupal 7 site structure - meaning content types and fields, views, contexts, and features. Theme development is happening in parallel with my data migration work below.
Why Migrate? What is a migration?
From the Migrate module page:
The migrate module provides a flexible framework for migrating content into Drupal from other sources (e.g., when converting a web site from another CMS to Drupal). Out-of-the-box, support for creating core Drupal objects such as nodes, users, and comments is included - it can easily be extended for migrating other kinds of content.”
And, from the main documentation page:
It is important to recognize that the Migrate module is a developer’s tool - if you do not have object-oriented programming skills in PHP, this module is not for you. The primary purpose is to provide the infrastructure for managing complex, large-scale site migrations.”
So, why use it for a Drupal upgrade? In certain situations it may make sense to start with a cleaner and more stable Drupal 7 site, and avoid any contrib module “lock-in” or upgrade path bugs. Migrate allows us to re-name content types or fields, and massage any content on its way into the new D7 site. Another compelling feature is the ability to roll back and re-run migrations; this feels much more efficient than starting over with the D6 site and running a full upgrade each time we need to change or try something. And somewhat related, you can re-run a migration without rolling back first and Migrate will only import the new content - awesome! Though I’ve barely scratched the surface with this exercise, Migrate has definitely proven to be powerful!
Setting up a new Migration
Since the Migrate V2 handbook pages are excellent, I’m just going to walk through the actual steps of creating a migration. This example will import Drupal users, keeping any role assignments (and even their user id’s). This is a real-world example, which I then modified slightly to import Blog content, Page content, and Taxonomy Terms. In each case, the destination “container” must already exist. So for this example, I have already created the roles in the new database (for terms, the vocabularies must exist). It’s also important to run the migrations in the correct order - I don’t want to import Blog posts before I have any users or terms to associate with them.
First, we need to create a custom migration module, to tell Drupal about our migration classes (the included migrate_example sub-module is a handy reference). Following the handbook page linked above, we’ll start with the .info file and a skeletal .module file:
Next, we’re going to define our d6 database at the top of the .module file:
For this to work, you need to alter your $databases array in settings.php:
Now head over to the modules page and enable the new migration module (it won’t do anything yet).
As stated on the Migration classes handbook page, we need to create the following objects, either by extending the classes provided by the Migrate module, or using them as-is:
- Source - define where the data is coming from
- Destination - define the Drupal 7 destinations for the data
- Mappings - link the Source and Destination so Migrate knows where to put the data
We’ll start by creating our_users.inc, to extend the generic Migrate class, override its constructor, and set up its group and description:
The parameter to the parent constructor is optional, and will define a migration group for this migration to belong to. From the handbook, “This may be useful if you have many migrations which fall into logical groups - say, a group for importing from a MySQL database, and another group operating off XML feeds.” Source
Continuing on, we define the Source class, using an SQL query and the conveniently included MigrateSourceSQL class (dig around in migrate/plugins/sources/ for other included sources):
The Migrate module includes a number of pre-defined destination classes (in migrate/plugins/destinations/) - and Migrate Extras adds even more. For this example, Migrate Extras is not required.
Phew, that was easy! The md5_passwords parameter, “Indicates whether incoming passwords are md5-encrypted - if so, we will rehash them similarly to the D6->D7 upgrade path.” Awesome, no need to force every user to change their password!
I doubt I can improve on the handbook description:
To track migration status, and allow rollback, the migrate module needs to remember what source record resulted in the creation of which destination object. It does this using a MigrateSQLMap class - this class creates database tables which map source and destination keys. To do this, it needs to know what data types the respective keys are. Thus, we pass in schema arrays declaring the respective source and destination keys. For the source key, since the migrate module knows nothing about the source table, you need to explicitly define the schema array (making sure that the key of the array, uid in this case, matches the field name used in the query passed to MigrateSourceSQL above). On the other hand, since we're using the canned class MigrateDestinationUser, it knows what the correct schema is already and can provide it through the static method getKeySchema().
The ‘machineName’ is used to create the migrate_map_ and migrate_message_ tables; passing in our migration class machine name should ensure that the table names are unique and easy to identify.
Next, we get to the actual data mappings (note that $destination is the first parameter, and $source is the second, though many of the field names are identical):
This ends our constructor function, but we still need to do a little work for the user roles. Enter prepareRow(). More details can be found on the Commonly implemented Migration methods handbook page, but the gist is that we can use prepareRow to modify data before it gets saved, or to conditionally skip importing a record (by returning FALSE). Keep in mind that records excluded in this manner will not be reflected on the Migrate admin page.
Running the Migration
There are two ways to interact with migrations: the Migrate UI or drush. In either case, you can do the following:
- Import - Perform one or more migration processes.
- Rollback - Roll back the destination objects from a given migration
- Rollback and Import - UI only, combines the two actions above
- Stop - Stop an active migration operation
- Reset - Reset a active migration’s status to idle
- Wipe - drush only, Delete all nodes from specified content types
There are also drush commands and UI pages to view source fields, destination fields, and field mappings. This is helpful for debugging and visualization, but you cannot assign any mappings through the UI or drush; they need to be in the php class. Migrate Resources
The Migrate handbook pages are great, but sometimes nothing replaces some working example code. Luckily, Drupal 6 to Drupal 7 migrations are a pretty common use case, and I found the resources below to be super helpful!
- Example module used in this post
- umbrella Migrate handbook page on drupal.org
- Implementing Migrate V2 handbook page
- Migrating content by using the migrate module - btmash
- Upgrade from Drupal 6 to Drupal 7 with Migrate 2 - xdeb - great example code; helped me a lot!
- Mike Ryan’s Drupal-to-Drupal data migration sandbox module - this looks cool, and from a great source, but I haven’t tried it personally.
Just a Start
As mentioned above, this is a real world example from a Drupal 6 to 7 migration which included Users, Terms, Pages, Blogs, and a custom Project content type. To minimize unforeseen issues and mapping work, we decided to keep all user, term, and node id’s the same (this may not be practical or desirable in all cases). This “upgrade” process using the Migrate module definitely feels more solid to me than a standard, “in-place” Drupal upgrade (for reasonably complex sites). And as a nice side-affect, you can import new content at the end of the dev cycle by simply running the migrations again, before the final launch.
Hopefully this helps you get started with more complex Drupal migrations using the Migrate module. Please leave a comment if anything needs clarification!