(draft) blog post about turbogears 2 packaging
authorStefano Zacchiroli <zack@upsilon.cc>
Sat, 30 May 2009 20:27:48 +0000 (22:27 +0200)
committerStefano Zacchiroli <zack@upsilon.cc>
Sat, 30 May 2009 20:27:48 +0000 (22:27 +0200)
blog/posts/2009/05/kick-starting_turbogears_2_packaging.mdwn [new file with mode: 0644]
blog/posts/2009/05/kick-starting_turbogears_2_packaging/get-orig-source.mk [new file with mode: 0644]
blog/posts/2009/05/kick-starting_turbogears_2_packaging/setup.py [new file with mode: 0755]

diff --git a/blog/posts/2009/05/kick-starting_turbogears_2_packaging.mdwn b/blog/posts/2009/05/kick-starting_turbogears_2_packaging.mdwn
new file mode 100644 (file)
index 0000000..bfa9090
--- /dev/null
@@ -0,0 +1,143 @@
+# TurboGears 2 in debian ... soon !
+
+After a long incubation, a few days ago [**TurboGears
+2**](http://turbogears.org/2.0/) has been released. Historically, I've
+been preferring TurboGears over [Django](http://www.django.org) for
+being closer to the open source philosophy of *reusing existing
+components*.
+
+Since the long 2.0 release was marking a gap with Django, I was eager
+to test the 2.0, and I was delighted to find it [in
+Debian](http://packages.debian.org/sid/python-turbogears2). Unfortunately,
+it [doesn't seem to be in good shape
+yet](http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=530879). In
+particular it **lacks several dependencies** before it can even be
+used to `quickstart` a project, in spite of its presence in unstable.
+
+To give an idea of the needed work, after having installed it manually
+(via `easy_install`), my virgin
+`/usr/local/lib/python2.5/site-packages/` has been polluted by **26
+egg-thingies**. I decided to spend some week-end time to start closing
+the gap, because we cannot lack such an important web framework in
+Debian (and ... erm, yes, also because I need it :-) ).
+
+Here is the **current status** of what has been ITP-ed / packaged
+already (by yours truly):
+
+* [ToscaWidgets](http://pypi.python.org/pypi/ToscaWidgets/),
+  [ITP](http://bugs.debian.org/531042) : in experimental/NEW,
+  packaging resurrected from a never-uploaded version
+* [repoze.tm2](http://pypi.python.org/pypi/repoze.tm2)
+  [ITP](http://bugs.debian.org/531038) : in experimental/NEW
+* [transaction](http://pypi.python.org/pypi/transaction/) (no ITP) :
+  [being packaged by the Zope
+  team](http://lists.debian.org/debian-devel/2009/05/msg00845.html)
+  (temporary unofficial package made by me available)
+* [repoze.who](http://www.repoze.org)
+  [ITP](http://bugs.debian.org/531083): in experimental/NEW
+* [repoze.who-{friendlyforms,plugins.sa,testutil}](http://pypi.python.org/pypi?%3Aaction=search&term=repoze.who&submit=search)
+  [ITP](http://bugs.debian.org/531146)
+  <small>(collected together (with a couple of others), as the
+  `python-repoze.who-plugins` package)</small> : in experimental/NEW
+* [repoze.what](http://pypi.python.org/pypi/repoze.what/)
+  [ITP](http://bugs.debian.org/531045) : in experimental/NEW
+* [repoze.what-{pylons,quickstart,plugins.sql}](http://pypi.python.org/pypi?%3Aaction=search&term=repoze.what&submit=search)
+  <small>(collected together (with a couple of others), as the
+  `python-repoze.what-plugins` package)</small> : just
+  [ITP](http://bugs.debian.org/531208)-ed
+
+## Where to
+
+Considering, foolishly, the last package as being already done, the
+way to go is still long:
+
+        zack@usha:~$ ls /usr/local/lib/python2.5/site-packages/|grep -v repoze.what
+        AddOns-0.6-py2.5.egg
+        BytecodeAssembler-0.3-py2.5.egg
+        Catwalk-2.0.2-py2.5.egg
+        easy-install.pth
+        Extremes-1.1-py2.5.egg
+        PEAK_Rules-0.5a1.dev_r2582-py2.5.egg
+        prioritized_methods-0.2.1-py2.5.egg
+        sprox-0.5.5-py2.5.egg
+        sqlalchemy_migrate-0.5.2-py2.5.egg
+        SymbolType-1.0-py2.5.egg
+        tg.devtools-2.0-py2.5.egg
+        tgext.admin-0.2.4-py2.5.egg
+        tgext.crud-0.2.4-py2.5.egg
+        TurboGears2-2.0-py2.5.egg
+        tw.forms-0.9.3-py2.5.egg
+        WebFlash-0.1a9-py2.5.egg
+        zope.sqlalchemy-0.4-py2.5.egg
+
+Possibly, some of them are already in Debian hidden somewhere, but
+sure there is still work to be done. **If you want to help**, you are
+more then welcome. The rules are simple: all packages will be
+maintained under the umbrella of the Python Modules Team, but you
+should be willing to take responsibility as the main packager.
+
+Please get in touch with if you are interested, as I'm in turn already
+in touch with some other very kind volunteers (thanks
+[Enrico](http://www.enricozini.org/blog/index.html) and Federico2!) to
+coordinate who-is-doing-what.
+
+## Preview packages available
+
+What has already been packaged, including a temporary workaround for
+`python-transaction`, will be available in experimental after NEW
+processing. The idea is that nothing will go to unstable until
+TurboGears 2 will be (proven to be) fully functional.
+
+In the meantime, **packages are available** from my personal APT
+repository (signed by my key), here are the friendly
+`/etc/apt/sources.list` :
+
+        deb http://people.debian.org/~zack/debian zack-unstable/
+        deb-src http://people.debian.org/~zack/debian zack-unstable/
+
+Versions are tilde-friendly and shouldn't get in your way when the
+official packages will hit unstable.
+
+## Packaging multiple-egg / multiple-upstream packages
+
+In all this, I've faced an interesting problem with the
+`python-repoze.{who,what}-plugins` packages. They correspond to a
+handful of plugins, each of which is about 20/30 Kb. I didn't consider
+appropriate to prepare 5 different packages, due to archive bloat
+potential. Sure you got the usual problem of multiple-upstream
+packages, but I consider it the right choice in this case.
+
+To counter some of the usual annoyances, as well as some
+[egg](http://peak.telecommunity.com/DevCenter/EggFormats)-specific
+annoyances, I wrote a couple of helpers:
+
+- [[get-orig-source.mk]] <small>([live
+  version](http://svn.debian.org/wsvn/python-modules/packages/python-repoze.who-plugins/trunk/debian/get-orig-source.mk))</small>:
+  a
+  `debian/rules` snippet which relies on several `debian/*.watch`
+  files which are fed to uscan to implement (not yet completely) the
+  `get-orig-source` target [described in
+  policy](http://www.debian.org/doc/debian-policy/ch-source.html/gret)
+
+- [[setup.py]] <small>([live
+  version](http://svn.debian.org/wsvn/python-modules/packages/python-repoze.who-plugins/trunk/setup.py))</small>:
+  a multiplexer for sub-directories shipping distutils-enabled Python
+  modules. That way you can use
+  `/usr/share/cdbs/1/class/python-distutils.mk` as usual, and setup.py
+  will take care of the "multiple upstream" problem at build time
+
+I'm no Python-packaging-guru so, lazyweb, if you spot in this choice
+something I utterly overlooked, or if you have improvements to
+suggest, please let me know.
+
+## Repacking eggs
+
+An interesting problem will be faced in trying to integrate the above
+approach with Python modules that are shipped only as `.egg` files
+(i.e., no tarballs, ... yes there are such horrifying things out
+there). To preserve uniformity, we would need `uscan` to support
+`--repack` ing of .egg files as they were simple .zip archives.
+
+Since eggs *are* .zip archives ... why not?
+
+[[!tag lang/english planet/debian python turbogears debian draft]]
diff --git a/blog/posts/2009/05/kick-starting_turbogears_2_packaging/get-orig-source.mk b/blog/posts/2009/05/kick-starting_turbogears_2_packaging/get-orig-source.mk
new file mode 100644 (file)
index 0000000..e8c8052
--- /dev/null
@@ -0,0 +1,34 @@
+# debian/rules snippet for creating multi-upstream tarball relying on
+# several debian/*.watch files
+
+# Copyright © 2009 Stefano Zacchiroli <zack@debian.org>
+# License: GNU GPL version 3 or above
+
+# TODO: does not yet fully implement policy wrt get-orig-source:
+#       - the target cannot be invoked from any dir (rely on "debian/")
+#       - tmpdir handling is not fully safe (e.g., no "mktemp -d")
+
+GOSTMP = get-orig.source.tmp
+ORIGNAME = $(PKG)-$(shell date +%Y%m%d)
+ORIGTARBALL = $(PKG)_$(shell date +%Y%m%d).orig.tar.gz
+
+get-orig-source: $(patsubst %,%/get-orig-source,$(UPSTREAMS))
+       cd $(GOSTMP) && \
+       mkdir $(ORIGNAME) && \
+       for p in $(UPSTREAMS) ; do \
+               mv `readlink $$p` $(ORIGNAME) ; \
+               mv $$p $(ORIGNAME) ; \
+       done && \
+       tar cvzf $(ORIGTARBALL) $(ORIGNAME) && \
+       mv $(ORIGTARBALL) ../ && \
+       cd .. && \
+       rm -rf $(GOSTMP)/
+$(GOSTMP):
+       -mkdir $@
+%/get-orig-source: $(GOSTMP)
+       uscan --watchfile debian/$*.watch \
+               --upstream-version 0 --package $* \
+               --download --destdir $(GOSTMP)/
+       cd $(GOSTMP) && \
+       tar xzf $**.orig.tar.gz && \
+       ln -s `tar tzf $**.orig.tar.gz | head -n1` $*
diff --git a/blog/posts/2009/05/kick-starting_turbogears_2_packaging/setup.py b/blog/posts/2009/05/kick-starting_turbogears_2_packaging/setup.py
new file mode 100755 (executable)
index 0000000..4a238f8
--- /dev/null
@@ -0,0 +1,24 @@
+#!/usr/bin/python
+# Multiplexer for invoking multiple setup.py in subdir
+# Copyright (C) 2009 Stefano Zacchiroli <zack@debian.org>
+# License: GNU GPL version 3 or above
+import os, string, sys
+
+if not os.environ.has_key('SUBDIRS') or not os.environ['SUBDIRS']:
+    print >> sys.stderr, "Can't find subdirs, please set SUBDIRS envvar"
+    sys.exit(3)
+else:
+    subdirs = os.environ['SUBDIRS'].split()
+setup_cmd = "python setup.py %s" % string.join(sys.argv[1:])
+
+topdir = os.getcwd()
+for d in subdirs:
+    if not os.path.isdir(d):
+        print >> sys.stderr, "WARNING: can't find subdir %s" % d
+        continue
+    os.chdir(d)
+    retcode = os.system(setup_cmd)
+    if retcode:
+        print >> sys.stderr, "ERROR: setup.py in subdir %s failed" % d
+        sys.exit(retcode >> 8)
+    os.chdir(topdir)