From 4f40dc60c125ad1e14071c234b67315fab6c6fc3 Mon Sep 17 00:00:00 2001 From: Rob Funk Date: Fri, 18 Jun 2004 04:17:07 +0000 Subject: Move shipper to dist-tools directory svn path=/trunk/; revision=3889 --- shipper/shipper.xml | 752 ---------------------------------------------------- 1 file changed, 752 deletions(-) delete mode 100644 shipper/shipper.xml (limited to 'shipper/shipper.xml') diff --git a/shipper/shipper.xml b/shipper/shipper.xml deleted file mode 100644 index 765f0f54..00000000 --- a/shipper/shipper.xml +++ /dev/null @@ -1,752 +0,0 @@ - - - -shipper -1 - - - shipper -automatic drop-shipping of project releases - - - - - shipper - -h - -n - -N - -f - -v - - - buildrpms - tarball - - - -Description - -shipper is a tool for shipping -project releases. Its job is to make it possible for you to run the -command shipper in the top-level directory of a -project and have a release be properly exported to all the places that -you normally deliver it — your personal website, Linux source -code archive sites, and distribution submission queues. A second goal -is to arrange your shipping process in such a way that metadata like -your project version only have to be kept in one place and modified -once per release. The overall goal is to reduce the friction cost -of shipping releases to as near zero as possible. - -buildrpms is a helper script that -builds source and binary RPMs from a specified tarball with a -BuildRoot field. shipper also calls - -rpm2lsm -1 to do part of its work. - -As much as possible, shipper tries to -deduce what it should do rather than requiring you to tell it. In -order to do this, it relies on your project obeying standard GNU-like -naming conventions. It also relies on being able to mine project -metadata out of a package specfile. (Presently the only variety of -package specfile supported is an RPM spec; this may change in the future, -when we fully support shipping Debian packages.) - -In normal use, you need set only one configuration variable, -which is the list of private destinations to ship to. You may also -want to add some magic Keywords comments to your -project specfiles. Once you have shipper -up and running, you can experiment with more advanced features -such as having the program generate project web pages for you. - - -Theory of Operation - -shipper pushes -deliverables out to -destinations. Deliverables include: source tarballs, -source zip archives, source RPMs, binary RPMs, ChangeLog files, README -files, LSM files, and various other project metadata files. Destinations -include both private destinations like websites, FTP -archive sites and mailing lists, and public -channels like ibiblio, freshmeat.net, and the submission -queues for various well-known operating-system distributions. The -shipper framework is extensible and it is relatively easy to add new -channel types and new deliverables; in the future, we hope to support -(for example) Debian packages as deliverables and SourceForge as a -channel. - -shipper's first step is to find the -project name and version, then to check that the minimum set of files that -shipper requires to continue is in place. -To start with, shipper needs a source -tarball and a specfile. Once it knows those are in place, it -can extract various pieces of information it will need to do its -real work. It also reads in a handful of configuration variables. -The -N (nobuild) option causes it to dump all configuration values and -stop there. - -The first real work that gets done is finding or building local -deliverables. These are either generated -deliverables (like RPMs) that can be rebuilt automatically, -or or stock deliverables (like a README file) -that have to be changed by hand. shipper -rebuilds any generated deliverable that doesn't exist when it starts -up. Building local deliverables is separated from uploading because -it means that you can stop and inspect what you're going to ship -before committing to an upload. - -The -n (noupload) option stops before uploading, leaving all -local deliverables in place but displaying the exact upload commands -that would have been used to ship them. The -f (force) option forces -a rebuild of all generated deliverables, even those that already -exist. The command shipper -f -n will show you -exactly what shipper would do for a real -upload. - -Once all local deliverables have been built, -shipper can begin uploading files and -posting announcements. It does private destinations first, then public -channels. This means, for example, that if you give -shipper your personal website as a destination, the -website will get updated each time before -any submissions or announcements are sent to public sites like -ibiblio.org or freshmeat.net. - -When uploads are complete, shipper -cleans up after itself by deleting any deliverables it created for -this run. Deliverables that were found and up to date are not -removed. - -Finally, note that shipper makes one -important assumption about the structure of your website(s). Beneath -each directory in your destinations list, there -will be one subdirectory for each project, with the directory leaf -name being the same as the project. Thus, for example, if you have -three projects named ruby, diamond and sapphire, and your personal -site is at gemstones.net:/public/www/precious/, -shipper will expect to be able to drop -deliverables in three directories -gemstones.net:/public/www/precious/ruby, -gemstones.net:/public/www/precious/diamond/, and -gemstones.net:/public/www/precious/sapphire/. -Note that shipper will not create these -project directories for you if they're missing; this is deliberate, so -that uploads to sites that are not prepared for them will fail -noisily. - - - -How Shipper Deduces What To Do - -The behavior of shipper depends on a handful of internal -variables. Some of these variables have defaults computed at startup -time. All can be set or overridden in the per-user -~/.shipper file, and overridden in any -per-project .shipper file. Both files are Python -code and the syntax of variable settings is Python's. - -If a variable is set in a config file, that value is locked in -(except for the destinations variable which can be -appended to from a specfile, see below) Variables that are -not set in a config file may be set by the values -of fields in your project specfile. - -For basic use, it is only necessary to set one such variable: -destinations, the list of destinations to ship to. -Normally you'll set this globally, pointing all your projects at your -main distribution website, in your ~/.shipper -file; it is also possible to add destinations on a per-project basis -by giving a comma-separated list in a #Destinations: comment in the -specfile. You can set the variable in a per-project -.shipper to ignore your global destination -list. - -The first thing shipper looks for is a specfile in the -current directory; there must be exactly one. It extracts the project -name from the Name field. Next step is to find the project version -(the variable package). This is extracted from the -specfile, or by looking for a makefile macro with a name -beginning with VERS; if the value of that macro is a shell command -wrapped in $(shell ...), it is executed and the output is captured to -yield the version. If both versions are present, they are -consistency-checked. - -shipper gets most of the rest of the -data it uses to decide what to do from headers in the specfile. -The following table lists all the variables and their corresponding -specfile fields. shipper uses the RPM spec -file fields: the Debian entries are informational only. - - - - - -Variable -RPM specfile field -Debian specfile field -Meaning - - - - -destinations -#Destinations: -XBS-Destinations: - -A list of remote directories to ship to using - -scp 1 -. Each location is a place to drop deliverables: -either a [user@]site:path destination that - -scp 1 - can use, or an FTP url that - -lftp 1 - -can use. Note that actual project directories are computed by -appending the value of package to -the destination you're shipping to. - -There is no default.. If you -do not set this variable, shipper will -ship only to public channels. - - - -channels -- -- - -The list of public channels to be shipped to after the private -channels in the destination list. You can disable -one or more of these in a config file by calling the function -disable(); for example with -disable('freshmeat'). - - - -whoami -- -- - -A plausible email address for the user. If not specified in the -config file, it's generated from -$USERNAME and $HOSTNAME. - - - -date -- -- - -The program's startup time. This can be used in the web page and -email announcement templates. - -You can use the Python function time.strftime("...") in your -~/.shipper file to format this date to your -taste. If you don't set this in the config file, the program will -set it for you. - - - -indextemplate -- -- - -Template HTML from which to generate index.html for shipping. There is a -default which generates a very simple page containing a title, a -date, and a table listing downloadable resources. This is used when -shipping to a web directory, if no index page exists when shipper -is run. - - - -mailtemplate -- -- - -Template text from which to generate the file ANNOUNCE.EMAIL to be -shipped to destinations that are mailto URLs. There is a default which -generates a very simple email containing a subject, a pointer to the -project web page, and the last entry in the project changelog. - - - -package -Name: -Package: - -Project name, used to generate the stem part of the names of RPMs and -other deliverables that shipper -builds. If the specfile is a Debian control file, the Debian-specific -part of the version number (after the dash) is removed. - - - -version -Version: -Version: - -Project version, used in generating the names of RPMs and -other deliverables that shipper -builds. - - - -homepage -URL: -XBS-Home-Page: - -Project home page URL. Used when generating project -announcements. - - - -arch -BuildArch: -Architecture: - -Build architecture. If this field is noarch, -noarch rather than binary RPMs will be built. - - - -keywords -#Keywords: -XBS-Keywords: - -Topic keywords. Used when generating LSM files. - - - -freshmeat_name -#Freshmeat-Name: -XBS-Freshmeat-Name: - -Freshmeat shortname, used in generating freshmeat.net -announcements. If this isn't present, it defaults to the project -name; you only need to set it if they differ. - - - -summary -Summary -Description: - -The one-line project summary field from your specfile. - - - -description -%description -Description: - -The Description field from your specfile. - - - -changelog -ChangeLog or %changelog -- - -If a ChangeLog file exists in the project -directory, its entire contents. Otherwise, if it exists, -the entire changelog section from the specfile. - - - -lastchange -ChangeLog or %changelog -- - - -If the source of your changlog was your specfile, this is the -most recent entry from your changelog without -its date/author/release header. If the source was Changelog, a -line of text directing the user to see the ChangeLog file. -This becomes the Changes field in your freshmeat.net announcement, -and freshmeat.net doesn't like the bulleted format of GNU ChangeLog -entries. - - - -resourcetable - - - -The HTML table of links to downloadable resources. This -variable is only computed if the index page is built. Any setting -of it in the startup files is ignored. - - - - - - -All these variables are available for substitution at the time a -web page or email announcement is generated. In general, any variable -you set in your ~/.shipper file will be available -at the time the web page or email announcement is generated. Use the -Python "%(variable)s" syntax, not shell-substitution syntax. - - - -Finding and Building Local Deliverables - -The following files are considered stock deliverables and may be -shipped if they are present when shipper -starts up: - - - - - -File -Explanation - - - - -README - -Project roadmap file. - - - -tarball - -The current source tarball, that is the file named ${package}-${version}.tar.gz. - - - -zipfile - -The current source zip archive, that is the file named ${package}-${version}.zip. - - - -NEWS - -Project news file. - - - -ChangeLog - -Project change log. - - - -HISTORY - -Project history file. - - - -BUGS - -Project bug list. - - - -TODO - -Current to-do list. - - - -*.html - -Any files with an .html extension will be shipped to all -website destinations. - - - - - - -Here are the generated deliverables that -shipper will build and ship, if they don't -exist when it starts up. Any of these that are created will be -deleted after a successful upload. - - - - - -Type -Explanation - - - - -index.html - -An index web page, to be shipped to any website destination. - - - -RPMs - -Source and either binary or noarch RPMs. - - - -LSM - -If the ibiblio channel is enabled, -shipper will generate a Linux Software Map -file for it. - - - -CHANGES - -If there is no ChangeLog file but there was a %changelog in your -specfile, shipper will generate a CHANGES -from the changelog entries in the specfile and ship that. - - - -ANNOUNCE.FRESHMEAT - -If there is no ANNOUNCE.FRESHMEAT file, -shipper will generate one. It will be a -job card that can be fed to freshmeat.net's XML-RPC interface via -freshmeat-submit1. - - - - -ANNOUNCE.EMAIL - -If there is no ANNOUNCE.EMAIL file, shipper -will generate one to be emailed to destinations that are mailto URLs. - - - - - - - -Shipping to Destinations -In operation, shipper walks through a -list of destinations, building the required deliverables for each one and -performing the required shipping actions to push them out to the -destination. Here are the channel types -shipper knows about: - - - - - - - - - -Channel Type -Deliverables -Specified by -Explanation - - - - -ibiblio -tarball, RPMs, LSM file -- - -If the ibiblio channel is enabled (it is by default), -shipper will attempt to ship a source -tarball, RPMs, and an an LSM file to ibiblio.org via FTP. The LSM -file will be automatically generated. - - - -redhat -RPMs -- - -If the Red Hat channel is enabled (it is by default), -shipper will attempt to ship source -and binary RPMs to the Red Hat submission directory via FTP. - - - -freshmeat -ANNOUNCE.FRESHMEAT -- - -If the freshmeat channel is enabled (it is by default), -shipper will attempt to post a release -announcement on freshmeat.net using -freshmeat-submit1. -The -announcement will include URLs for whichever of the following -deliverables are shipped, using the URL field from your specfile: tarball, -zipfile, RPMs, CHANGES. The user will be -prompted for a Freshmeat release-focus. This announcement is -generated into the local deliverable ANNOUNCE.FRESHMEAT. - - - -Generic Web site -README, tarball, zipfile, RPMs, CHANGES, NEWS, HISTORY, *.html, -BUGS, TODO. -scp destination ([user@]host:dir) - -This channel type represents a website. -shipper uses -scp1 -to put deliverables on websites. If the user part of the scp -destination is absent, it will be taken from the environment variable -USERNAME. - -No generic Web sites are shipped to by default. You must declare -them by putting scp destinations in the destinations -variable. - - - -Generic FTP site -tarball, RPMs -FTP URL - -Old-fashioned FTP site with no metadata. The FTP URL is parsed -to get the sitename and directory where deliverables should be dropped. The -FTP username to be used will be taken from the environment variable -USERNAME. The FTP password will be looked up in your -~/.netrc file. - -No generic FTP sites are shipped to by default. You must -declare them by putting FTP urls in the -destinations variable. - - - -Email address -ANNOUNCE.EMAIL -mailto URL - -The contents of the generated ANNOUNCE.EMAIL file is emailed to -each email address specified as a channel. - -No email channels are set up by default. You must -declare them by putting mailto: urls in the -destinations variable. - - - -rsync unit -SRPM -rsync address ([user@]host::unit) - -An SRPM is shipped to each destination that is rcognized as -an rsync address (by the double colon). - -No rsync channels are set up by default. You must -declare them by putting rsync addresses in the -destinations variable. - - - - - - - -Command-line Options - -The -n option of shipper suppresses -uploads, just building all deliverables locally. The -N option -suppresses both uploads and builds, generating a configuration dumop -instead. The -f option forces rebuilding of local deliverables even -if they already exist. The -v option makes -shipper chatty about what it's doing. The --h option prints a usage message and exits. - - - -Hints and Tips -The following variable definition in your makefile will ensure -that the makefile version is derived from (and thus always consistent -with) the specfile version. - - -VERS=$(shell sed <*.spec -n -e '/Version: \(.*\)/s//\1/p') - - -A makefile production like the following will allow -you to type make release and be sure that all -the deliverables shipper knows about -will be rebuilt before being shipped. - - -release: package-$(VERS).tar.gz package.html - shipper -f - - -You will want to change package to your -project name. Note that you should not use this recipe if your -project has its own (non-generated) index page, as the -f option will -overwrite index.html. - -To make - -rpm1 - -build noarch rather than binary RPMs, insert the following header in -your specfile: - - -BuildArch: noarch - - - -Author -Eric S. Raymond esr@thyrsus.com. The buildrpms -script was originally by Sean Reifschneider. - -There is a project web page at -http://www.catb.org/~esr/shipper/. - - -Bugs -The rsync channel type is untested. Shipping Debian packages -should be supported. - - -See Also - - - -freshmeat-submit -1 -, - -lftp -1 -, - -rpm2lsm -1 -, - -scp -1 -, - -ssh -1 -. - - - - -- cgit v1.2.3