Re-roll patches to apply cleanly to the latest version of Drupal

Ready for review -- Last peer review: 20 Feb 2015
Description: 
Learn to update or "re-roll" patches to apply to a newer version of Drupal
Overview: 

In the time it takes to get a patch written, tested, reviewed, etc. Drupal continues evolving. As Drupal evolves, the patches must be kept up to date. For example, imagine a patch that was originally created to make some edits to line 5 in file X. Two releases later the patch should actually edit line 10, not line 5. Now the patch has to be updated, or "re-rolled". Sometimes re-rolling a patch is complex, but usually it's pretty simple. Git provides some handy tools that will let you apply the patch to the version of Drupal it was written for, merge the changes into a newer version, and then re-create the patch.

Prerequisites: 
Steps: 

Before getting started, it helps to have a basic understanding of what we're about to do with Git. Here's a brief overview of what's going on in Git during the patch re-rolling process.

Every saved revision of a Git project is called a 'commit'. Every commit in a project's history has an ID number (Figure 1.1).

When a new Drupal release comes out the project is tagged with the release number, for example,7.4, 7.5, 7.6, 7.7. A tag is just an alias for a commit ID. It is a human readable name assigned to a specific revision of the project. (Figure 1.2).

Before re-rolling a patch, we will find a place in the project's history where the patch applies cleanly. (Figure 1.3)

Next we will create a local working branch, or 'topic' branch, based on a version of the project where our patch applies cleanly. (Figure 1.4)

Then we merge in more recent changes from branch 7.x to our working branch and resolve any conflicts that arise during the merge. (Figure 1.5)

Finally, we re-roll the patch by getting the diff between our local branch and Drupal's development branch. Then we save the new diff as a patch and post it on drupal.org.

(Figure 1.6 shows all the previous diagrams combined into one, in case it's helpful for reference.)

How To:

  1. Go to the Drupal core issue queue and find patches that need to be reviewed by filtering the list of issues tagged as "Needs reroll".

    We will use the example in https://www.drupal.org/patch/reroll which is based onthis issue. This example may not work at the time you're reading this, but it will help you to familiarize with rerolling patches.
  2. Make sure you're running the latest version of Drupal
    cd /path/to/drupal
    git pull
  3. Find a patch that was created a while ago. Download and try to apply the patch.
    git apply --check 1497310-statistics_config_settings-5.patch
    • If it applies cleanly, add a comment to the issue stating that "the patch applied cleanly" and remove the tag Needs Re-roll. Then go back to step 1.
    • If Git reports errors when trying to apply the patch, the patch needs to be re-rolled.
  4. Next we need to get the patch to apply cleanly. Go to the issue and note the date of the comment that you downloaded the patch from. (Double click on the date to toggle between time-ago and the date.)

    git log -1 --before="March 30, 2012".

  5. Take the first few characters of the commit hash you see there (in this case, 72cf5a755) and checkout the code from that commit. checkout -b means to create a new local branch and automatically switch to it. In the following example, cmi-statistics will become the name of the new branch. You can choose a different name.
    git checkout -b cmi-statistics 992692441
  6. Try applying the patch to test if it works:
    git apply --index 1497310-statistics_config_settings-5.patch
    If you see no output, you're good to go. If you get similar "patch does not apply" errors, delete your current branch and repeat the previous step with an earlier commit hash (e.g. e179ee1d6 in this case).
  7. Commit the patch's changes to your local branch, run git status to see these changes. Avoid adding files which weren't in the initial branch, like the patch.
    Then, commit the changes
    git commit -m "Applying patch from issue 1497310 comment 5804956"
  8. Now, attempt to pull in all of the changes that have happened since the commit you branched from. Any rows that begin with "CONFLICT" are the ones you'll need to sort out in order for the patch to apply cleanly.
    git rebase 8.0.x

  9. Go through each of the files listed as having conflicts, and look for a line like <<<<<<< HEAD. For example:
    Open core/modules/statistics/statistics.module file

    ...
    /**
    * Implements hook_cron().
    */
    function statistics_cron() {
    <<<<<<< HEAD
    $statistics_timestamp = \Drupal::state()->get('statistics.day_timestamp') ?: 0;
    =======
    $config = config('statistics.settings');
    $statistics_timestamp = $config->get('statistics_day_timestamp');
    >>>>>>> Applying patch from issue 1497310 comment 5804956

    Everything above the ======= is from the "clean" upstream version. Everything below the ======= is found in the patch you are rerolling. In general, you want to keep what's in the clean upstream version, and then modify it to follow the changed behavior from the patch. This ensures that any other changes that have been made to the upstream version not affecting your patch do not accidentally get undone.

  10. Edit the file until it looks correct, making sure to remove the special conflict marker lines.
    For example:

    function statistics_cron() {
    $config = config('statistics.settings');
    $statistics_timestamp = $config->get('statistics_day_timestamp');
  11. Stage your changes with
    git add
    (Be careful not to accidentally stage files that have been removed in HEAD. Use git rm for those if necessary.)
  12. In order to continue rebase process run:
    git rebase --continue
    and repeat the previous steps until there are no conflicts left to resolve.
  13. Finally, create your patch, by diffing your local branch (cmi-statistics) against the upstream branch:
    git diff 8.0.x cmi-statistics > [description]-[issue-number]-[comment-number].patch
  14. Test your patch by applying it to the upstream branch.
    git checkout 8.0.x
    git apply --check [description]-[issue-number]-[comment-number].patch

    If the patch apply cleanly, you've just re-rolled a patch. Congratulations!.

Comments

Hiya. Great tutorial. Just wanted to point out that on http://drupal.org/patch/reroll they suggest to use 'git rebase' instead of 'git merge'. It was confusing to me, initially, which instructions to follow.

Took me a while and a lot of Googling to figure out that they both end up essentially with the same result, although rebase gives you one commit, and merge gives you two: your original commit + a merge commit.

Hi. Two suggestions:

(1) x-posting from my comment in core-office hours. When rerolling a patch, STEP 9 **IS REALLY IMPORTANT**. Otherwise you'll have a patch with missing stuff. :D

(2) Perhaps more detail on STEP 8 would be hepful. It might also be a good idea to introduce people to gui-based 3-way merge tools or to 'git mergetool'.

Before doing this thing i thing that a person has to qualified engineer then h/she can do this one.

New Version of drupal is amazing but it is really typical. I am thinking to learn it properly so I took help with assignment UK for better guidance.

I have been at it for hours, but I cannot find a patch that patches cleanly at all. Ihave tried to detective steps for multiple patches and no commit ID seems to work. Is there something I am missing? No patch I find seems to have a commit ID with it. I feel like maybe I don't know where to look for it