Moving data to Drupal 8

Moving Up

Migration Groups

The first step is to create a migration group. Migration groups are configuration entities provided by Migrate Plus. The configuration management system in Drupal 8 uses configuration entities to import and export configurations between different environments. Configuration entities are defined in a YAML file. The group configuration should be placed in the config/install/ directory of a custom module. For the pet migrations, a pets group is created and placed in config/install/migrate_plus.migration_group.pets.yml:

id: pets
label: Pet migrations
description: A few simple pet migrations.source_type: Custom tables
source:
key: migrate

The group has a machine id and human readable label and description. The source_type key contains a human-readable description of the data source. In this case, the source is a custom database table. Additional keys may be added to the group configuration and will be shared by all migrations that are part of the group. The source[key] field defines the source database connection. This key must exist in the $databases array in settings.php. For example:

$databases = array(
  'default' => array(...),
  'migrate' => array(
    'default' => array(
      'database' => 'pets',
      ...
    ),
  ),
);

You can organize migrations into as many groups as necessary and import an entire group or individual entities.

Migration Configurations

The next step is to create migration configurations. Migrations are also created as configuration entities provided by Migrate Plus. They tell Drupal about the source and destination types and provide field mappings. Migration configurations should be placed in the config/install/ directory.

The pet owner migration is defined in config/install/migrate_plus.migration.pet_owners.yml, and the pet migration is defined in config/install/migrate_plus.migration.pets.yml. The module migrate_pets file structure is shown in Figure 5, and the full pet migration from config/install/migrate_plus.migration.pets.yml is shown in Listing 1.

Listing 1

Pet Migration

id: pets
label: "Friendly, Furry Pets"
migration_group: pets
source:
  plugin: pet_source
destination:
  plugin: "entity:node"
process:
  nid: pid
  title: name
  type:
    plugin: default_value
    default_value: pet
  uid:
    plugin: migration
    migration: pet_owners
    source: oid
  field_pet_type:
    plugin: cat_to_dog
  field_photo: picture
migration_dependencies:
  required:
     - pet_owners
dependencies:
  enforced:
    module:
      - migrate_pets
Figure 5: Migration configurations in config/install.

Like the group configuration, each migration has a machine id and a human-readable label. These are used in migration lists and will be included throughout the UI when available. This migration is part of the pets group, as specified by the migration_group key. The group is optional, and the migration will be part of a default group if it is omitted.

Source Plugins

The source key selects a plugin for loading source data. The Migrate Drupal module in Drupal core provides many default source plugins for Drupal 6 and 7, including d6_node, d7_user, and others. The best way to discover a source plugin is to look in the corresponding module source code. For example, the d6_node plugin is located in the core node module at /core/modules/node/Plugin/migrate/source/d6/Node.php. In this case, you use a custom source plugin to read from the pet database (pet_source):

source:
  plugin: pet_source

The source plugin is placed in the custom module at src/Plugin/migrate/source/PetSource.php and extends \Drupal\migrate\Plugin\migrate\source\SqlBase (for non-database sources, the plugin should extend \Drupal\migrate\Plugin\migrate\SourcePluginBase). The source plugin is exposed to Drupal using an @MigrateSource annotation. The annotation id field (pet_source) is referenced as the source plugin in the migration configuration:

/**
 * Pet source from database.
 *
 * @MigrateSource(
 *   id = "pet_source"
 * )
 */
class PetSource extends SQLBase {

At a minimum, source plugins must implement the MigrateSourceInterface::fields() and MigrateSourceInterface::getIds() methods. The fields() method returns an array of fields available for mapping (Listing 2).

Listing 2

@MigrateSource

/**
 * {@inheritdoc}
 */
public function fields() {
  return [
    'pid' => $this->t('Pet id'),
    'oid' => $this->t('Owner id'),
    'type' => $this->t('Pet type'),
    'name' => $this->t('Pet name'),
    'picture' => $this->t('Pet photo'),
  ];
}
/**
 * {@inheritdoc}
 */
public function getIds() {
  return ['pid' => ['type' => 'integer']];
}

Data ID fields are not necessarily migrated one-to-one in Drupal (pid from the source might not be the same pid in the destination). Instead, Drupal keeps a map of source IDs to destination IDs to track changes. The getIds() method returns the unique ID field and schema type for the source. In this example, each pet has a unique integer ID.

SQL sources like the pet database must implement the SqlBase::query() method to define a select query used to load a source row. Use $this->select() (Listing 3) to query the source database configured in the migration (remember the shared source[key] field in the group configuration).

Listing 3

Query the Source Database

/**
 * {@inheritdoc}
 */
public function query() {
  $query = $this->select('pet', 'p')
    ->fields('p', ['pid', 'oid', 'type', 'name',
      'picture'])
    ->condition('picture', 'IS NOT NULL');
  return $query;
}

You can optionally override MigrateSourceInterface::prepareRow() to make changes to row values before they are passed to field mappings. Most data manipulation should be done with process plugins in the field mapping, but prepareRow() is useful for lower-level manipulations, such as unserialize() or changing data types. In this example, we convert pet pictures to animations. Assume the PetSource::animate() method is implemented and converts a static image to an animated GIF. Row values are fetched using $row->getSourceProperty() and set with $row->setSourceProperty() (Listing 4).

Listing 4

Fetch and Set Row Values

/**
 * {@inheritdoc}
 */
public function prepareRow(Row $row) {
  if ($picture = $row->getSourceProperty('picture')) {
    $row->setSourceProperty('picture',
      $this->animate($picture));
  }
  return parent::prepareRow($row);
}

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • WordPress, Joomla, and Drupal compared
    Open source content management systems have an abundance of extensions scattered across the web. We compare the features of three CMSs against various consumer requirements.
  • Drush vs. Drupal Console
    Let's take a stroll through the brave new world of Drupal 8, now with Drupal Console; lowering the learning curve for the newbie and the seasoned pro alike.
  • Unprecedented built-in support for diverse languages
    Thanks to the efforts of the Drupal 8 Multilingual Initiative, you can install Drupal in at least 100 languages and exercise granular control over the languages on your website.
  • Moving your data – It's not always pretty
    The world is swimming in data, and the pool is getting deeper at an alarming rate. At some point, you will have to migrate data from one set of storage devices to another. Although it sounds easy, is it? We take a look at some tools that can help.
  • Migration from LDAP to FreeIPA
    The change from centralized user authentication on a vanilla LDAP server to the FreeIPA identity management solution is easier than many admins think. Given attention to a few points, the migration takes very little time and effort.
comments powered by Disqus