Development/Packaging/Tools/repsys
From Mandriva
Repository System Tool
This page will explain how to setup the repsys tool. Basic usage examples are shown elsewhere. Work in progress, please do not edit yet
Contents |
Introduction
The repository system tool (repsys for short) was developed to help manage RPM repositories stored in SVN. It allows one to submit packages, obtain an SRPM, convert commit messages to %changelog format and much more. Below we will see in details what this tool can do in server and client mode.
The repsys package is available in the contrib repository.
Repsys as a server
Repsys is used in server mode in the XML RPC part of the buildsystem:
- checkout the SOURCES and SPECS directories of the given package at the revision the client requested;
- create a %changelog section and append it to the spec file;
- optionally run an external rebrand program which can, for example, choose a release tag (like %mkrel) or change the spec file in any other way;
- run rpm -bs to construct a .src.rpm file;
- place this .src.rpm file in a directory specified in /etc/repsys.conf for this target.
Installation steps
- Read this documentation fully
- Install the repsys rpm package and its dependencies;
- Install apache;
- If using a rebrand script, create it
- Configure /etc/repsys.conf
- Configure apache authentication
- Configure apache to execute the XML RPC CGI script
Detailed explanations are below.
The /etc/repsys.conf configuration file
This file is used both in server and in client mode. Here is a sample:
[global] verbose = no default_parent = svn+ssh://svn.mandriva.com/svn/packages/cooker/ [log] oldurl = svn+ssh://svn.mandriva.com/svn/packages/misc/ [users] andreas = Andreas Hasenack <[email protected]> oeriksson = Oden Eriksson <[email protected]> [submit] name = cooker target = /buildsystem/cooker allowed = svn+ssh://svn.mandriva.com/svn/packages/cooker/ scripts = /svnroot/cooker.admin/rebrand-cooker.sh
Below is an explanation of these options:
- default_parent: URL to prepend to the package name when no url is provided. This is mostly used in client mode. For example, with the above configuration file, both commands below would end up accessing the same URL:
repsys getsrpm -l kolab repsys getsrpm -l svn+ssh://svn.mandriva.com/svn/packages/cooker/kolab
- oldurl: this URL has the old %changelogs from the pre-SVN "era". When packages are imported into this SVN scheme, the %changelog tag has to be removed because we start using the SVN commit messages instead. In order to not loose these old %changelog entries, they are stored at this URL, one for each package. When repsys is building a .src.rpm, it appends this old changelog to the spec file;
- [users] section: contains the svn login to email mapping used to create %changelog entries. Since SVN only stores the login name of the person who made the commit, this mapping has to be used to write out a compliant %changelog of the form Name <email>. Currently this list has to be maintained manually at the server, but I'm sure this could be easily expanded to use some sort of LDAP lookup, for example. If the SVN commit logs contains a username which is not listed here, then only the username will be used in the %changelog (i.e., no email part);
-
[submit] section: contains the definition of all targets supported by this installation. Each target has the following parameters:
- name: the target name. Used as the -t argument to the repsys submit client command;
- target: whitespace-separated list of directories indicating where the .src.rpm file has to be placed. For example, to submit packages simultaneously to ia32 and x86_64 buildsystems one could use /buildsystem/cooker-ia32 /buildsystem/cooker-x86_64 here;
- allowed: URLs pointing to which SVN repositories can have packages submitted through this target;
-
scripts: if used, contains a whitespace-separated list of scripts that will be called with three parameters, in this order:
- topdir: rpm's topdir, i.e., where SOURCES and SPECS are;
- specfile: the spec filename of the package being processed;
- svn-revision: the SVN revision that was extracted
Sample rebrand script
The rebrand script (parameter scripts in the [submit] section) can be used to modify the SPEC file after it was extracted from SVN and already had a %changelog section appended. This modification is only local and will only appear in packages which went through the buildsystem: nothing is commited back to SVN.
One use we found for such a script in the Conectiva case was to automatically use the SVN revision number of the last commit (obtained from the already added %changelog section, see below for an example) as an RPM Release tag:
#!/bin/bash
TOPDIR=$1
SPEC=$2
REV=$3
# Use svn revision as release
tac $SPEC | perl -pe '/^\+ [\d-]+ [\d:]+ \((\d+)\)/ && ($r = $1); s/^(Release:).*/$1 ${r}mdk/;' | tac > ${SPEC}.tmp;
mv -f $SPEC{.tmp,}
To better see what this sample does, consider the following %changelog section which was automatically created by repsys and appended to the SPEC file:
%changelog * Fri Aug 26 2005 Andreas Hasenack <[email protected]> + 2005-08-26 09:36:08 (749) - rebuild and recover changelog which was mistakenly erased in the previous package * Fri Aug 26 2005 Andreas Hasenack <[email protected]> + 2005-08-25 19:13:47 (743) - Compile with -fPIC to fix python binding, see #17246 (fix by Couriousous <[email protected]>) (...)
With this %changelog, the shown rebrand script would change the Release tag in the SPEC file to be Release: 749mdk.
This was just an example of the kind of transformations that can be done at this stage before the .src.rpm package is created and sent to the build daemon/machine.
The XML RPC CGI script
The XML RPC CGI script is the actual repsys "server". It's the part that listens to the repsys submit client command and creates the .src.rpm file that is to be sent to the build system/daemon.
This is the actual CGI script:
/svnroot/mdv.scripts/xmlrpc:
#!/usr/bin/python from RepSys.cgi import xmlrpcserver import tempfile tempfile.tempdir="/var/tmp" xmlrpcserver.main()
This script is then called as a CGI from Apache when the repsys command line tool is used to submit a package. The following Apache configuration can be used to enable script execution and to authenticate and authorize the user:
/etc/httpd/conf/webapps.d/repsys.conf:
ScriptAlias /scripts/mdv "/svnroot/mdv.scripts/"
<Directory "/svnroot/mdv.scripts">
AllowOverride All
Options ExecCGI
<LimitExcept GET>
AuthType Basic
AuthName "Authorization Realm"
AuthUserFile /svnroot/mdv.admin/svn.passwd
Require valid-user
</LimitExcept>
<IfModule mod_access.c>
Order allow,deny
Allow from all
</IfModule>
</Directory>
This means read-only repsys client commands can access the repository without authentication, but a package submission and other write operations must be authenticated.
The web submission form
Packages can also be submitted via a web interface if so desired. The repsys package contains such a script. Sample screenshot:
#!/usr/bin/python from RepSys.cgi import submit import tempfile tempfile.tempdir="/var/tmp" submit.main()
The same Apache configuration we used for the xmlrpc server is used again, which means only authenticated users can submit packages.
Repsys as a client
Repsys has many uses at the client side, the most important one being of course the package submission. This section will show the most used client side commands.
Most commands take as an argument a repository URL. Many users find it convenient to use a shell variable which points to the repository root and then just append the package name. For example:
~/.bashrc:
export REPOS="svn+ssh://svn.mandriva.com/svn/packages/cooker"
This can be used in a checkout, for example:
svn co $REPOS/bm/current bm
Package submission: submit
Please see Repository System Quickstart -- Submitting for an example.
%changelog output: rpmlog
The rpmlog repsys command converts the SVN commit log messages to the RPM %changelog format. This is how the .src.rpm packages from the buildsystem end up with a proper %changelog section even though the SPEC file itself in SVN doesn't have one.
If you just want to get a proper .src.rpm with a %changelog entry, however, then you may prefer to use the getsrpm command shown further below.
Example
[andreas@pandora smart]$ repsys rpmlog svn+ssh://svn.mandriva.com/svn/packages/cooker/smart > changelog.txt [andreas@pandora smart]$ head changelog.txt * Wed Oct 05 2005 Andreas Hasenack <[email protected]> + 2005-10-05 20:57:36 (990) - updated to version 0.40 - releasing 0.40-1mdk * Tue Sep 20 2005 Helio Chissini de Castro <[email protected]> + 2005-09-20 10:31:05 (922) - Reenabled gpg signatire check ( Thanks Andreas ) * Mon Sep 12 2005 Andreas Hasenack <[email protected]>
SRPM package extraction: getsrpm
Syntax: repsys getsrpm [-l] <package-url>
Example: repsys getsrpm -l svn+ssh://svn.mandriva.com/svn/packages/cooker/smart
This command is used to obtain an .src.rpm file with data stored in SVN. This file is then stored in the current working directory.
A parameter which is used very often is -l. With this parameter, the %changelog section is added to the spec file prior building the .src.rpm package.
There are many other parameters used to checkout specific versions of a package, please see the output of repsys getsrpm --help.
Example
[andreas@pandora ~]$ mkdir smart [andreas@pandora ~]$ cd smart [andreas@pandora smart]$ repsys getsrpm -l svn+ssh://svn.mandriva.com/svn/packages/cooker/smart [andreas@pandora smart]$ l smart-0.40-1mdk.src.rpm -rw-r--r-- 1 andreas andreas 544K Out 6 11:37 smart-0.40-1mdk.src.rpm [andreas@pandora smart]$ rpm -qp --changelog smart-0.40-1mdk.src.rpm | head * Qua Out 05 2005 Andreas Hasenack <[email protected]> + 2005-10-05 20:57:36 (990) - updated to version 0.40 - releasing 0.40-1mdk * Ter Set 20 2005 Helio Chissini de Castro <[email protected]> + 2005-09-20 10:31:05 (922) - Reenabled gpg signatire check ( Thanks Andreas ) * Seg Set 12 2005 Andreas Hasenack <[email protected]>
