From Mandriva Community Wiki

General view of the existing machine, environment, scripts and procedure to build the Mandriva distribution

Contents 1 Fetching information 1.1 Checking if new versions are available 1.2 Getting the sources 1.3 Current packages version 2 Development 2.1 Build machines 2.2 Development tools 3 Building 3.1 Manual building 3.2 Automatic building 4 Uploading 4.1 Developer submit 4.2 Upload process 5 Mirroring 5.1 Local mirroring 5.2 External mirroring 6 Maintainers 7 Monitoring

Fetching information

Checking if new versions are available

The update report from youri-check demo run by a third-party display new versions available from various sources.

Getting the sources

If you proceed manually, you can use any client (wget, lftp, curl...) to download sources. The alternative is to use mdvsys or repsys update, that does it automatically.

Current packages version

The mandriva subversion repository hold everything needed to build the packages: spec files, sources, patches.

Development

See:

• RPM Howto
• Maintainer Howto

Build machines

The build machines, namely 'the compilation cluster', is a set of x86 32 bit and x86 64 bit machines. Any official Mandriva maintainer has an access to all these machines. The home directories are spread over the machines through NFS, which involves performances penalties when building on any other node than the one holding the home directory locally.

The machines run a stable version of Mandrivalinux, and include a chroot of the development version, cooker, where the developers are working. The machines also include old stable development chroot for support and maintenance of old distributions.

The machines are accessible via a gateway, kenobi, which is holding also a local mirror of the distributions.

Development tools

They are basically two redundant toolsets available:

• repsys (SVN interactions) + bm or rpmbuild (building)
• mdvsys

The first one is the historical one, inherited from conectiva build system, and is implemented in python. The second one has been reimplemented in perl, by contributors. Both toolsets include similar set of features. Choosing either one, or both, is definitively a personal choice.

Building

In the current architecture, the final build should be performed by a rebuild bot after that the source package is validated. In some critical case, the build can happen on the cluster machines, for various reasons (missing buildrequires, short-circuit, lack of time). Only a few admin of the Mandriva Community can force packages upload without automatic rebuild.

Manual building

Automatic building

Two bots are used to rebuild the distributions, Iurt to rebuild the packages and sync two architectures, and rpm-rebuilder to recompile the whole distribution.

Uploading

Developer submit

Developers can use a collection of scripts to submit their package. The core script is mdvsys, associated with youri-submit.

Upload is a more sophisticated script which is compiling the packages and also performing several checks before submitting the package.

Upload process

Once the packager has submitted a package, it is copied on a temporary queue on kenobi, then Ulri the compilation scheduler will send packages on the build host so Iurt can rebuild them. Once compiled, the packages are moved in another queue, waiting for the distribution server to fetch them and to remove them from kenobi. Once on the distribution server, the package is checked, and then eventually put into cooker or rejected.

When a source package is uploaded into the distribution, a mail is sent to the changelog mailing list, when a package is rejected, a mail is sent to the maintainers mailing list and the owner of the package. The rejected packages are kept for some time.

The frequency of the upload process is directly linked to the time needed to upload a package. Initially an upload consisted of :

• checking the package
• committing the package into the SVN
• putting the package in the repository
• rebuilding the hdlist
• mirroring to the compilation cluster so that the developers can use it
• mirroring to the mirrors reference machine, where external public mirrors will fetch it

Now committing to the SVN and mirroring to the external mirror reference machine have been made asynchronous, so developers don't have to suffer from the network trouble between Mandriva, SVN server and the mirror reference machine.

Mirroring

Local mirroring

The distribution server is mirroring twice per hour cooker to a local machine, after a successful upload, so that the hdlists are correct.

External mirroring

This local machine then mirrors cooker to the external mirror machine, so that public mirrors can get the packages from it.

Maintainers

The maintainer database can be accessed here

Monitoring

The youri-check demo run by a third-party offers a generic view of some packages problems.