path: root/dist-tools/shipper/shipper.1

.\"Generated by db2man.xsl. Don't modify this, modify the source.
.de Sh \" Subsection
.br
.if t .Sp
.ne 5
.PP
\fB\\$1\fR
.PP
..
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Ip \" List item
.br
.ie \\n(.$>=3 .ne \\$3
.el .ne 3
.IP "\\$1" \\$2
..
.TH "SHIPPER" 1 "" "" ""
.SH NAME
 shipper \- automatic drop-shipping of project releases
.SH "SYNOPSIS"

.nf
\fBshipper\fR [-h] [-n] [-N] [-f] [-v]
.fi

.nf
\fBbuildrpms\fR {\fItarball\fR}
.fi

.SH "DESCRIPTION"

.PP
shipper is a tool for shipping project releases. Its job is to make it possible for you to run the command \fBshipper\fR in the top-level directory of a project and have a release be properly exported to all the places that you normally deliver it -- your personal website, Linux source code archive sites, and distribution submission queues. A second goal is to arrange your shipping process in such a way that metadata like your project version only have to be kept in one place and modified once per release. The overall goal is to reduce the friction cost of shipping releases to as near zero as possible.

.PP
buildrpms is a helper script that builds source and binary RPMs from a specified tarball with a BuildRoot field. shipper also calls \fBrpm2lsm\fR(1) to do part of its work.

.PP
As much as possible, shipper tries to deduce what it should do rather than requiring you to tell it. In order to do this, it relies on your project obeying standard GNU-like naming conventions. It also relies on being able to mine project metadata out of a package specfile. (Presently the only variety of package specfile supported is an RPM spec; this may change in the future, when we fully support shipping Debian packages.)

.PP
In normal use, you need set only one configuration variable, which is the list of private destinations to ship to. You may also want to add some magic ``Keywords'' comments to your project specfiles. Once you have shipper up and running, you can experiment with more advanced features such as having the program generate project web pages for you.

.SH "THEORY OF OPERATION"

.PP
shipper pushes deliverables out to destinations. Deliverables include: source tarballs, source zip archives, source RPMs, binary RPMs, ChangeLog files, README files, LSM files, and various other project metadata files. Destinations include both private destinations like websites, FTP archive sites and mailing lists, and public channels like ibiblio, freshmeat.net, and the submission queues for various well-known operating-system distributions. The shipper framework is extensible and it is relatively easy to add new channel types and new deliverables; in the future, we hope to support (for example) Debian packages as deliverables and SourceForge as a channel.

.PP
shipper's first step is to find the project name and version, then to check that the minimum set of files that shipper requires to continue is in place. To start with, shipper needs a source tarball and a specfile. Once it knows those are in place, it can extract various pieces of information it will need to do its real work. It also reads in a handful of configuration variables. The -N (nobuild) option causes it to dump all configuration values and stop there.

.PP
The first real work that gets done is finding or building local deliverables. These are either generated deliverables (like RPMs) that can be rebuilt automatically, or or stock deliverables (like a README file) that have to be changed by hand. shipper rebuilds any generated deliverable that doesn't exist when it starts up. Building local deliverables is separated from uploading because it means that you can stop and inspect what you're going to ship before committing to an upload.

.PP
The -n (noupload) option stops before uploading, leaving all local deliverables in place but displaying the exact upload commands that would have been used to ship them. The -f (force) option forces a rebuild of all generated deliverables, even those that already exist. The command \fBshipper -f -n\fR will show you exactly what shipper would do for a real upload.

.PP
Once all local deliverables have been built, shipper can begin uploading files and posting announcements. It does private destinations first, then public channels. This means, for example, that if you give shipper your personal website as a destination, the website will get updated each time before any submissions or announcements are sent to public sites like ibiblio.org or freshmeat.net.

.PP
When uploads are complete, shipper cleans up after itself by deleting any deliverables it created for this run. Deliverables that were found and up to date are not removed.

.PP
Finally, note that shipper makes one important assumption about the structure of your website(s). Beneath each directory in your \fIdestinations\fR list, there will be one subdirectory for each project, with the directory leaf name being the same as the project. Thus, for example, if you have three projects named ruby, diamond and sapphire, and your personal site is at \fIgemstones.net:/public/www/precious/\fR, shipper will expect to be able to drop deliverables in three directories \fIgemstones.net:/public/www/precious/ruby\fR, \fIgemstones.net:/public/www/precious/diamond/\fR, and \fIgemstones.net:/public/www/precious/sapphire/\fR. Note that shipper will not create these project directories for you if they're missing; this is deliberate, so that uploads to sites that are not prepared for them will fail noisily.

.SH "HOW SHIPPER DEDUCES WHAT TO DO"

.PP
The behavior of shipper depends on a handful of internal variables. Some of these variables have defaults computed at startup time. All can be set or overridden in the per-user \fI~/.shipper\fR file, and overridden in any per-project \fI.shipper\fR file. Both files are Python code and the syntax of variable settings is Python's.

.PP
If a variable is set in a config file, that value is locked in (except for the \fIdestinations\fR variable which can be appended to from a specfile, see below) Variables that are not set in a config file may be set by the values of fields in your project specfile.

.PP
For basic use, it is only necessary to set one such variable: \fIdestinations\fR, the list of destinations to ship to. Normally you'll set this globally, pointing all your projects at your main distribution website, in your \fI~/.shipper\fR file; it is also possible to add destinations on a per-project basis by giving a comma-separated list in a #Destinations: comment in the specfile. You can set the variable in a per-project \fI.shipper\fR to ignore your global destination list.

.PP
The first thing shipper looks for is a specfile in the current directory; there must be exactly one. It extracts the project name from the Name field. Next step is to find the project version (the variable \fIpackage\fR). This is extracted from the specfile, or by looking for a makefile macro with a name beginning with VERS; if the value of that macro is a shell command wrapped in $(shell ...), it is executed and the output is captured to yield the version. If both versions are present, they are consistency-checked.

.PP
shipper gets most of the rest of the data it uses to decide what to do from headers in the specfile. The following table lists all the variables and their corresponding specfile fields. shipper uses the RPM spec file fields: the Debian entries are informational only.
VariableRPM specfile fieldDebian specfile fieldMeaning\fIdestinations\fR#Destinations:XBS-Destinations:
.PP
A list of remote directories to ship to using \fBscp\fR(1). Each location is a place to drop deliverables: either a [user@]site:path destination that \fBscp\fR(1) can use, or an FTP url that \fBlftp\fR(1) can use. Note that actual project directories are computed by appending the value of \fIpackage\fR to the destination you're shipping to.

.PP
There is no default.. If you do not set this variable, shipper will ship only to public channels.
\fIchannels\fR--
.PP
The list of public channels to be shipped to after the private channels in the \fIdestination\fR list. You can disable one or more of these in a config file by calling the function \fBdisable()\fR; for example with \fBdisable('freshmeat')\fR.
\fIwhoami\fR--
.PP
A plausible email address for the user. If not specified in the config file, it's generated from \fB$USERNAME\fR and \fB$HOSTNAME\fR.
\fIdate\fR--
.PP
The program's startup time. This can be used in the web page and email announcement templates.

.PP
You can use the Python function time.strftime("...") in your \fI~/.shipper\fR file to format this date to your taste. If you don't set this in the config file, the program will set it for you.
\fIindextemplate\fR--
.PP
Template HTML from which to generate index.html for shipping. There is a default which generates a very simple page containing a title, a date, and a table listing downloadable resources. This is used when shipping to a web directory, if no index page exists when shipper is run.
\fImailtemplate\fR--
.PP
Template text from which to generate the file ANNOUNCE.EMAIL to be shipped to destinations that are mailto URLs. There is a default which generates a very simple email containing a subject, a pointer to the project web page, and the last entry in the project changelog.
\fIpackage\fRName:Package:
.PP
Project name, used to generate the stem part of the names of RPMs and other deliverables that shipper builds. If the specfile is a Debian control file, the Debian-specific part of the version number (after the dash) is removed.
\fIversion\fRVersion:Version:
.PP
Project version, used in generating the names of RPMs and other deliverables that shipper builds.
\fIhomepage\fRURL:XBS-Home-Page:
.PP
Project home page URL. Used when generating project announcements.
\fIarch\fRBuildArch:Architecture:
.PP
Build architecture. If this field is ``noarch'', noarch rather than binary RPMs will be built.
\fIkeywords\fR#Keywords:XBS-Keywords:
.PP
Topic keywords. Used when generating LSM files.
\fIfreshmeat_name\fR#Freshmeat-Name:XBS-Freshmeat-Name:
.PP
Freshmeat shortname, used in generating freshmeat.net announcements. If this isn't present, it defaults to the project name; you only need to set it if they differ.
\fIsummary\fRSummaryDescription:
.PP
The one-line project summary field from your specfile.
\fIdescription\fR%descriptionDescription:
.PP
The Description field from your specfile.
\fIchangelog\fRChangeLog or %changelog-
.PP
If a \fIChangeLog\fR file exists in the project directory, its entire contents. Otherwise, if it exists, the entire changelog section from the specfile.
\fIlastchange\fRChangeLog or %changelog-
.PP
If the source of your changlog was your specfile, this is the most recent entry from your changelog without its date/author/release header. If the source was Changelog, a line of text directing the user to see the ChangeLog file. This becomes the Changes field in your freshmeat.net announcement, and freshmeat.net doesn't like the bulleted format of GNU ChangeLog entries.
\fIresourcetable\fR--
.PP
The HTML table of links to downloadable resources. This variable is only computed if the index page is built. Any setting of it in the startup files is ignored.

.PP
All these variables are available for substitution at the time a web page or email announcement is generated. In general, any variable you set in your \fI~/.shipper\fR file will be available at the time the web page or email announcement is generated. Use the Python "%(variable)s" syntax, not shell-substitution syntax.

.SH "FINDING AND BUILDING LOCAL DELIVERABLES"

.PP
The following files are considered stock deliverables and may be shipped if they are present when shipper starts up:
FileExplanationREADME
.PP
Project roadmap file.
tarball
.PP
The current source tarball, that is the file named ${package}-${version}.tar.gz.
zipfile
.PP
The current source zip archive, that is the file named ${package}-${version}.zip.
NEWS
.PP
Project news file.
ChangeLog
.PP
Project change log.
HISTORY
.PP
Project history file.
BUGS
.PP
Project bug list.
TODO
.PP
Current to-do list.
*.html
.PP
Any files with an .html extension will be shipped to all website destinations.

.PP
Here are the generated deliverables that shipper will build and ship, if they don't exist when it starts up. Any of these that are created will be deleted after a successful upload.
TypeExplanationindex.html
.PP
An index web page, to be shipped to any website destination.
RPMs
.PP
Source and either binary or noarch RPMs.
LSM
.PP
If the ibiblio channel is enabled, shipper will generate a Linux Software Map file for it.
CHANGES
.PP
If there is no ChangeLog file but there was a %changelog in your specfile, shipper will generate a CHANGES from the changelog entries in the specfile and ship that.
ANNOUNCE.FRESHMEAT
.PP
If there is no ANNOUNCE.FRESHMEAT file, shipper will generate one. It will be a job card that can be fed to freshmeat.net's XML-RPC interface via \fBfreshmeat-submit\fR(1).
ANNOUNCE.EMAIL
.PP
If there is no ANNOUNCE.EMAIL file, shipper will generate one to be emailed to destinations that are mailto URLs.

.SH "SHIPPING TO DESTINATIONS"

.PP
In operation, shipper walks through a list of destinations, building the required deliverables for each one and performing the required shipping actions to push them out to the destination. Here are the channel types shipper knows about:
Channel TypeDeliverablesSpecified byExplanationibibliotarball, RPMs, LSM file-
.PP
If the ibiblio channel is enabled (it is by default), shipper will attempt to ship a source tarball, RPMs, and an an LSM file to ibiblio.org via FTP. The LSM file will be automatically generated.
redhatRPMs-
.PP
If the Red Hat channel is enabled (it is by default), shipper will attempt to ship source and binary RPMs to the Red Hat submission directory via FTP.
freshmeatANNOUNCE.FRESHMEAT-
.PP
If the freshmeat channel is enabled (it is by default), shipper will attempt to post a release announcement on freshmeat.net using \fBfreshmeat-submit\fR(1). The announcement will include URLs for whichever of the following deliverables are shipped, using the URL field from your specfile: tarball, zipfile, RPMs, CHANGES. The user will be prompted for a Freshmeat release-focus. This announcement is generated into the local deliverable ANNOUNCE.FRESHMEAT.
Generic Web siteREADME, tarball, zipfile, RPMs, CHANGES, NEWS, HISTORY, *.html, 
BUGS, TODO.scp destination ([user@]host:dir)
.PP
This channel type represents a website. shipper uses \fBscp\fR(1) to put deliverables on websites. If the user part of the scp destination is absent, it will be taken from the environment variable \fBUSERNAME\fR.

.PP
No generic Web sites are shipped to by default. You must declare them by putting scp destinations in the \fIdestinations\fR variable.
Generic FTP sitetarball, RPMsFTP URL
.PP
Old-fashioned FTP site with no metadata. The FTP URL is parsed to get the sitename and directory where deliverables should be dropped. The FTP username to be used will be taken from the environment variable \fBUSERNAME\fR. The FTP password will be looked up in your \fI~/.netrc\fR file.

.PP
No generic FTP sites are shipped to by default. You must declare them by putting FTP urls in the \fIdestinations\fR variable.
Email addressANNOUNCE.EMAILmailto URL
.PP
The contents of the generated ANNOUNCE.EMAIL file is emailed to each email address specified as a channel.

.PP
No email channels are set up by default. You must declare them by putting mailto: urls in the \fIdestinations\fR variable.
rsync unitSRPMrsync address ([user@]host::unit)
.PP
An SRPM is shipped to each destination that is rcognized as an rsync address (by the double colon).

.PP
No rsync channels are set up by default. You must declare them by putting rsync addresses in the \fIdestinations\fR variable.

.SH "COMMAND-LINE OPTIONS"

.PP
The -n option of shipper suppresses uploads, just building all deliverables locally. The -N option suppresses both uploads and builds, generating a configuration dumop instead. The -f option forces rebuilding of local deliverables even if they already exist. The -v option makes shipper chatty about what it's doing. The -h option prints a usage message and exits.

.SH "HINTS AND TIPS"

.PP
The following variable definition in your makefile will ensure that the makefile version is derived from (and thus always consistent with) the specfile version.

.nf

VERS=$(shell sed <*.spec -n -e '/Version: \\(.*\\)/s//\\1/p')

.fi

.PP
A makefile production like the following will allow you to type \fBmake release\fR and be sure that all the deliverables shipper knows about will be rebuilt before being shipped.

.nf

release: package-$(VERS).tar.gz package.html 
	shipper -f

.fi

.PP
You will want to change package to your project name. Note that you should not use this recipe if your project has its own (non-generated) index page, as the -f option will overwrite \fIindex.html\fR.

.PP
To make \fBrpm\fR(1) build noarch rather than binary RPMs, insert the following header in your specfile:

.nf

BuildArch: noarch

.fi

.SH "AUTHOR"

.PP
Eric S. Raymond <esr@thyrsus.com>. The buildrpms script was originally by Sean Reifschneider.

.PP
There is a project web page at : http://www.catb.org/~esr/shipper/http://www.catb.org/~esr/shipper/.

.SH "BUGS"

.PP
The rsync channel type is untested. Shipping Debian packages should be supported.

.SH "SEE ALSO"

.PP
 \fBfreshmeat-submit\fR(1), \fBlftp\fR(1), \fBrpm2lsm\fR(1), \fBscp\fR(1), \fBssh\fR(1).