Migration Groups

The first step is to create a migration group. Migration groups are configuration entities provided by Migrate Plus. The configuration management system in Drupal 8 uses configuration entities to import and export configurations between different environments. Configuration entities are defined in a YAML file. The group configuration should be placed in the config/install/ directory of a custom module. For the pet migrations, a pets group is created and placed in config/install/migrate_plus.migration_group.pets.yml:

id: pets
label: Pet migrations
description: A few simple pet migrations.source_type: Custom tables
source:
key: migrate

The group has a machine id and human readable label and description. The source_type key contains a human-readable description of the data source. In this case, the source is a custom database table. Additional keys may be added to the group configuration and will be shared by all migrations that are part of the group. The source[key] field defines the source database connection. This key must exist in the $databases array in settings.php. For example:

$databases = array(
  'default' => array(...),
  'migrate' => array(
    'default' => array(
      'database' => 'pets',
      ...
    ),
  ),
);

You can organize migrations into as many groups as necessary and import an entire group or individual entities.

Migration Configurations

The next step is to create migration configurations. Migrations are also created as configuration entities provided by Migrate Plus. They tell Drupal about the source and destination types and provide field mappings. Migration configurations should be placed in the config/install/ directory.

The pet owner migration is defined in config/install/migrate_plus.migration.pet_owners.yml, and the pet migration is defined in config/install/migrate_plus.migration.pets.yml. The module migrate_pets file structure is shown in Figure 5, and the full pet migration from config/install/migrate_plus.migration.pets.yml is shown in Listing 1.

id: pets
label: "Friendly, Furry Pets"
migration_group: pets
source:
  plugin: pet_source
destination:
  plugin: "entity:node"
process:
  nid: pid
  title: name
  type:
    plugin: default_value
    default_value: pet
  uid:
    plugin: migration
    migration: pet_owners
    source: oid
  field_pet_type:
    plugin: cat_to_dog
  field_photo: picture
migration_dependencies:
  required:
     - pet_owners
dependencies:
  enforced:
    module:
      - migrate_pets

Figure 5: Migration configurations in config/install.

Like the group configuration, each migration has a machine id and a human-readable label. These are used in migration lists and will be included throughout the UI when available. This migration is part of the pets group, as specified by the migration_group key. The group is optional, and the migration will be part of a default group if it is omitted.

Source Plugins

The source key selects a plugin for loading source data. The Migrate Drupal module in Drupal core provides many default source plugins for Drupal 6 and 7, including d6_node, d7_user, and others. The best way to discover a source plugin is to look in the corresponding module source code. For example, the d6_node plugin is located in the core node module at /core/modules/node/Plugin/migrate/source/d6/Node.php. In this case, you use a custom source plugin to read from the pet database (pet_source):

source:
  plugin: pet_source

The source plugin is placed in the custom module at src/Plugin/migrate/source/PetSource.php and extends \Drupal\migrate\Plugin\migrate\source\SqlBase (for non-database sources, the plugin should extend \Drupal\migrate\Plugin\migrate\SourcePluginBase). The source plugin is exposed to Drupal using an @MigrateSource annotation. The annotation id field (pet_source) is referenced as the source plugin in the migration configuration:

/**
 * Pet source from database.
 *
 * @MigrateSource(
 *   id = "pet_source"
 * )
 */
class PetSource extends SQLBase {

At a minimum, source plugins must implement the MigrateSourceInterface::fields() and MigrateSourceInterface::getIds() methods. The fields() method returns an array of fields available for mapping (Listing 2).

/**
 * {@inheritdoc}
 */
public function fields() {
  return [
    'pid' => $this->t('Pet id'),
    'oid' => $this->t('Owner id'),
    'type' => $this->t('Pet type'),
    'name' => $this->t('Pet name'),
    'picture' => $this->t('Pet photo'),
  ];
}
/**
 * {@inheritdoc}
 */
public function getIds() {
  return ['pid' => ['type' => 'integer']];
}

Data ID fields are not necessarily migrated one-to-one in Drupal (pid from the source might not be the same pid in the destination). Instead, Drupal keeps a map of source IDs to destination IDs to track changes. The getIds() method returns the unique ID field and schema type for the source. In this example, each pet has a unique integer ID.

SQL sources like the pet database must implement the SqlBase::query() method to define a select query used to load a source row. Use $this->select() (Listing 3) to query the source database configured in the migration (remember the shared source[key] field in the group configuration).

/**
 * {@inheritdoc}
 */
public function query() {
  $query = $this->select('pet', 'p')
    ->fields('p', ['pid', 'oid', 'type', 'name',
      'picture'])
    ->condition('picture', 'IS NOT NULL');
  return $query;
}

You can optionally override MigrateSourceInterface::prepareRow() to make changes to row values before they are passed to field mappings. Most data manipulation should be done with process plugins in the field mapping, but prepareRow() is useful for lower-level manipulations, such as unserialize() or changing data types. In this example, we convert pet pictures to animations. Assume the PetSource::animate() method is implemented and converts a static image to an animated GIF. Row values are fetched using $row->getSourceProperty() and set with $row->setSourceProperty() (Listing 4).

/**
 * {@inheritdoc}
 */
public function prepareRow(Row $row) {
  if ($picture = $row->getSourceProperty('picture')) {
    $row->setSourceProperty('picture',
      $this->animate($picture));
  }
  return parent::prepareRow($row);
}