Rpmbuild and git
From Mandriva Community Wiki
The Old Way
Creating a patch
People are using a mix of different techniques. Here is a list:
- before modifying a file in BUILD/foo-version/, create a backup of the directory, then use diff -ur
- before modifying a file, create a backup of that file with a specific extension[1], then use gendiff. In case of file creation, this technique can be quite bad[1]
Comments on the patch can end-up in various places:
- in the patch filename (eg: fix-xxx-when-zzz-is-foobar.patch)
- as comments in spec file before PatchX: ... line
- in the %changelog (ie in the SVN commit log)
- added at the beginning of the patch
Most of these techniques discourage long commits.
Modifying a patch
- either you modify the patch by hand, but this is tedious if you change the number of lines
- or you use either editiff (or recountdiff in cases where editdiff fails)
- or you modify some files, then use gendiff using the extension .xxx used on the %patchX -p1 -b .xxx line. But the result will be wrong if the file was also modified by %patchY when Y > X
- or you modify foo.spec to stop after the patch you want to modify, then call rpmbuild -bp or bm -p. You can then modify some files, then use gendiff. This is quite burdensome (for example you must redo the %build step to be sure a C modification compiles).
Regenerating a patch with gendiff, the drawbacks
- comments at the beginning of the patch must be re-added by hand. You may loose these comments if you are not cautious enough.
- once again, in case of file creation, this technique can be quite bad[1].
Rediffing a patch
You first fix possible rejects[1]. patch can be misleading when displaying a patch is already applied (in case of patch on multiple files).
Then you usually use gendiff, see above for gendiff drawbacks when modifying a patch.
The git Way
Creating a git repository
To create a git repository from the tarballs + the patches, do one of the following:
- either call rpmbuild or bm with option --with git_repository
- or add %_with_git_repository 1 to your ~/.rpmmacros
And that's it! Let's see an example:
% mdvsys co autologin
Checking out package bash into directory autologin
% cd autologin
% grep Patch SPECS/autologin.spec
Patch0: autologin-1.0.0-mdv.patch
Patch1: autologin-glibc28_fix.diff
% bm -p --with git_repository
processing package autologin-1.0.0-%mkrel 24
running prep stage
succeeded!
% cd BUILD/autologin-1.0.0
% git log
commit 9f7ce457097a45fce89828eeeede2a331368fcb8
Author: unknown <>
Date: Thu Jan 22 15:48:35 2009 +0000
glibc28 fix
commit 305a5692a328f1a3ef43435c61be88c4eab77cf6
Author: unknown <>
Date: Wed May 21 09:40:56 2008 +0000
mdv
commit 7738ce758145e124d566bbc0c0c33588bdfd6645
Author: unknown <>
Date: Wed May 21 11:40:56 2008 +0200
imported autologin-1.0.0.tar.bz2
Creating a patch
Quite easy with git! All you need is git add[1], git commit and git format-patch. Example:
% emacs src/autologin.c % git commit -m 'set variable FOO as needed by bar' src/autologin.c Created commit c5f5143: set variable FOO as needed by bar 1 files changed, 1 insertions(+), 0 deletions(-) % git format-patch HEAD^ % git format-patch HEAD^ 0001-set-variable-FOO-as-needed-by-bar.patch % mv 0001-set-variable-FOO-as-needed-by-bar.patch ../../SOURCES/set-variable-FOO-as-needed-by-bar.patch % cd ../.. % emacs SPECS/autologin.spec % mdvsys sync % svn commit
Modifying a patch
Use git power!
- first commit the change (with a dumb commit log),
- then merge the change into an existing patch using git rebase -i vanilla. Git will ask you how to rebase your commits. Just move your latest commit just after commit corresponding to patch you want to regenerate and change its status from pick to squash.
- extract patch from git, either using git show or git format-patch.
Rediffing a patch
You must first create a git repository with the previous version:
% mdvsys co foo % cd foo % bm -p --with git_repository if you don't have git_url defined in the specfile (no upstream repository available) : % git clone --bare BUILD/foo-1.0.0 foo.git if you have git_url defined in the specfile, foo.git already exist : % cd foo.git % git remote add -f --mirror <a_remote_name> ../BUILD/foo-1.0.0 <a_remote_name> can be anything you want to identify old version, for instance : foo-1.0.0
Now you can modify the spec to adapt %define version to the new version. And then:
% bm -p --with git_repository # this fails because of some patches conflicts % cd BUILD/foo-1.0.1 % git checkout origin/patches-applied % git rebase --onto vanilla origin/vanilla if you have git_url defined : %git checkout
And now, you just have to handle conflicts using git :)
When the git rebase is finished, you can obtain all the patches using git format-patch vanilla
Cherry-picking from upstream git
If you add something like:
%define git_url git://git.savannah.gnu.org/libtool.git # or, with a specific branch: %define git_url git://rpm.org/rpm rpm-4.6.x
in your spec file, you will be able to use git cherry-pick.
Note that a temporary git clone --bare is done in %_topdir/%name.git, so that git will only download the full git repository the first time; on the next bm, only a git fetch is done to ensure the local git clone is up-to-date.
Macro %apply_patches
Instead of maintaining both PatchX and %patchX, you can replace all the "%patchX -p1" lines with %apply_patches[1].
conditional %patchX
You can not use %apply_patches when you have some conditional %patch. Example:
%ifarch %ix86 %patch99 -p1 %endif
A way to handle this is to move conditionality inside the patched file. For example by doing #if inside some C code.
If you really need to conditionally apply a patch, give up %apply_patches !
backports
If you need to backport your package on distributions prior to Mandriva Linux 2009.1, add the following to your spec file[1]:
%if 0%{?!apply_patches:1}
%define apply_patches for patch in $(awk '/^Patch.*:/ { print "%{_sourcedir}/"$2 }' %{_specdir}/%{name}.spec); do patch -s -p1 -i $patch; done
%endif
Packaging directly from git
(Alternate approach)
The goal of this procedure is to minimize the steps from tweaking the source code to making an RPM ready to be installed.
Warning : this method has not been reviewed nor approved by Mandriva. Use with care.
First-time build for a project
Step 0: Prepare your git-scripts (once only)
Prepare a directory where all the projects will lie (eg. ~/build/) and setup the git macros in ~/.rpmmacros accordingly. (Or use the procedure described above? )
# Add these to rpmmacros to get git functionality.
# %packager ...
%distsuffix xrg
# You may ask to also get this:
# %git_submodule ~/bin/git-submodule
%git_repodir %(echo ~/build/)
%git_gitdir %{git_repodir}/%{git_repo}/.git
%git_get_source pushd %{git_repodir}/%{git_repo} ;\
/usr/bin/git archive --format=tar --prefix=%{name}-%{version}/ %{git_head} | \
gzip -c > %{_sourcedir}/%{name}-%{version}.tar.gz ;\
popd
%git_clone_source if [ -d %{name}-%{version} ] ; then \
cd %{name}-%{version} && git pull origin %{git_head} ; \
else \
git clone %{git_gitdir} %{name}-%{version} && \
cd %{name}-%{version}/ ; \
fi
%git_prep_submodules %{git_submodule} init --cloned && %{git_submodule} update
%git_get_ver %(git --git-dir=%{git_gitdir} describe --tags | sed 's/^v\\?\\(.*\\)-\\([0-9]\\+\\)-g.*$/\\1/;s/-//')
%git_get_rel %(git --git-dir=%{git_gitdir} describe --tags | grep '\\-g.\\+$' | sed 's/^v\\?\\(.*\\)-\\([0-9]\\+\\)-g.*$/\\2/')
Step 1: Import the project
Case A: Project is already in a git Do nothing
Case B: Project has source code in tarball Recommended procedure is to put the source in a git directory in such a way that it will welcome future upgrades. Say, we have the tarball foobar-0.2.tar.gz which contains
foobar-0.2/COPYING foobar-0.2/README foobar-0.2/src/...
we could do:
cd ~/build/ tar -xzf ~/tmp/foobar-0.2.tar.gz mv foobar-0.2 foobar #we want to strip the ver. cd foobar git init #initialize the .git metainfo # do git config here, setup the committer name # setup the .gitignore so that junk doesn't get in git add . # Commit what was in the tarball git commit -m "Initial 0.2 release" git tag 'v0.2' # Mark the release
Case C: Project is in some other repository Use git svn, git cvsimport, git svnimport etc. to import the upstream project. For bzr-based projects, use tailor from darcs (note: a tweak is highly recommended to fix the commit string..)
Make sure that versions are tagged right and that the import procedure has been done right (cross check with some snapshot tarball, perhaps).
Step 2: Mark Mandriva branch
Simply, do:
git checkout -b Mandriva
This will allow you to put any Mandriva-specific fixes/files there.
Step 3: Commit the spec file.
Select an appropriate directory (like contrib/ doc/ addons/ ) depending on the project and place the mandriva spec file there.
Step 4: Tweak the spec file
Put a few lines to have it point the right git dir:
%define git_repo foobar #the name of the dir under ~/build/
%define git_head Mandriva # the branch we want to use
Name: foobar
Version: %{git_get_ver}
Release: %{git_get_rel}
and make sure the Source: is one and square:
Source0: %{name}-%{version}.tar.gz
(anything else would conflict with the %git_prep_source scriptlet)
Then, append one line before the setup:
%prep + %git_get_source %setup -q
( %setup should be that simple, too)
All these make sure that before you build, your latest source will come from the git repo.
Of course commit the spec at the end (although the git macros can work with a 'dirty' spec)
Step 5: Bootstrap the rpmbuild source
Rpmbuild cannot start if a source file is not there, so we need to fool it that the source had been there before the %git_.. macros:
echo | gzip -c > ~/rpmbuild/SOURCES/foobar-0.2.tar.gz
This will just be an empty (but with gz-header) file.
Step 6: Copy, or symlink, the spec file in the rpmbuild dir.
If you are doing many builds, symlink is better..
ln -s ~/build/foobar/contrib/foobar-mdv.spec ~/rpmbuild/SPECS/foobar.spec
Step 7: Build the RPM!
rpmbuild -ba ~/rpmbuild/SPECS/foobar.spec
This has been 7 steps already...
Second-time build
But the real beauty comes the second time you want to build the project:
Step b-1: Fix the bug
cd ~/build/foobar/src kwrite main.c
Step b-2: Commit the bug
git commit main.c
Step b-2.1: Test it
Given that you have got .gitignore right, any make you do in the ~/build/foobar won't affect the source files for the rpm. You can do all sorts of development there.
Step b-3: Build the RPM!
rpmbuild -ba ~/rpmbuild/SPECS/foobar.spec
See! No patches, no tars, no need to bump releases. You can now use git to manage the patches, or push them upstream, or revert them or whatever.
Package Maintenance
When foobar 0.3 is released, you need to do:
Step m-1: Go to the original version
cd ~/build/foobar git status #make sure it's clean git checkout master #the branch w/o Mandriva fixes
Step m-2: Checkout the new source
tar -xzf foobar-0.3.tar.gz --strip-components=1 # make sure you don't create ~/build/foobar/foobar-0.3 .. git status
Step m-3: Inspect the Source.
Inspect the changed files, ensure they are there, right over the old source.
Step m-4: Commit.
git add . # or src/*.c ... selectively git status #make sure everything is marked for commit. git commit -m 'Upgrade to ver 0.3' git tag 'v0.3'
Stem m-5: Go to Mandriva branch and merge
git checkout Mandriva git merge master
Here, git will try to re-apply Mandriva changes. If any patch fails, it will ask for manual intervention (with 'git mergetool').
Stem m-6: Build the rpm
rpmbuild -ba ~/rpmbuild/SPECS/foobar.spec
