From mark at klomp.org  Sun Jan  4 11:31:34 2009
From: mark at klomp.org (Mark Wielaard)
Date: Sun, 04 Jan 2009 20:31:34 +0100
Subject: serving forests over http
Message-ID: <1231097494.3380.39.camel@hermans.wildebeest.org>

Hi,

I was setting up a backup of the forests to provide some redundancy in
case the server goes down again. But I am unable to convince
hgwebdir.cgi (or actually hg serve) to provide forest support through
http. fclone locally or through ssh works fine though.
How did you configure hgwebdir to make fclone over http work?

Thanks,

Mark



From mark at klomp.org  Tue Jan  6 01:56:25 2009
From: mark at klomp.org (Mark Wielaard)
Date: Tue, 06 Jan 2009 10:56:25 +0100
Subject: hg repos/forests backups (Was: serving forests over http)
In-Reply-To: <1231097494.3380.39.camel@hermans.wildebeest.org>
References: <1231097494.3380.39.camel@hermans.wildebeest.org>
Message-ID: <1231235785.3426.28.camel@dijkstra.wildebeest.org>

Hi,

On Sun, 2009-01-04 at 20:31 +0100, Mark Wielaard wrote:
> I was setting up a backup of the forests to provide some redundancy in
> case the server goes down again. But I am unable to convince
> hgwebdir.cgi (or actually hg serve) to provide forest support through
> http. fclone locally or through ssh works fine though.
> How did you configure hgwebdir to make fclone over http work?

Thanks to Kelly I figured some stuff out.

There is now a mirror of the IcedTea repos (icedtea, icedtea6, brandweg,
fedora, icepick, pulseaudio and the testrepo), the main OpenJDK repos
(jdk6 and jdk7 master forests) and a copy of the old openjdk svn and
openjdk6 code drop hg mirrors at: http://classpath.wildebeest.org/hg/
This is a read-only mirror that is synced once an hour.

The main issues setting this up were:

- The forest repo itself has moved. There are two:
  http://hg.akoha.org/hgforest/ (the main repo)
  http://www.bitbucket.org/pmezard/hgforest-crew/
  (for making forest work with the latest and greatest - but see below)

- hgwebdir.config isn't a real hgrc file, only parts of it are parsed.
  (In particular [extensions] aren't parsed)

- Some versions of hgwebdir don't pick up the default hgrc files.
  You might have to play with something like:
  import os
  os.environ["HGRCPATH"] = "/var/lib/hg/.hgrc:/etc/mercurial/hgrc"
  (where one of those hgrc files has the forest extension defined)

- It seems that whatever version of forest you use, the hgweb/http
  support only reliably works with mercurial 0.95. I couldn't get it to
  work with anything newer or older (local file system fclones or
  fclones through ssh do work fine with newer versions though, just the
  http versions - also those created with hg serve - don't.)

- Unfortunately 0.95 hgwebdir doesn't work nicely with nested (forest)
  repos. You will end up having to define every forest in full in your
  hgwebdir.config like:

  [paths]
  jdk6/corba = /hg/jdk6/corba
  jdk6/hotspot = /hg/jdk6/hotspot
  jdk6/jaxp = /hg/jdk6/jaxp
  jdk6/jaxws = /hg/jdk6/jaxws
  jdk6/jdk = /hg/jdk6/jdk
  jdk6/langtools = /hg/jdk6/langtools

  Which is a pity since with 1.1.2 you could just do:
  [paths]
  / = /hg/**
  (all assuming you have a top level /hg dir with all your repos)

- Mercurial 1.1 and higher support the fncache repo format, Mercurial
  0.95 won't be able to read these. So if you experimented with 1.1+
  then you will have to convert your repo first:
  http://www.selenic.com/mercurial/wiki/index.cgi/fncacheRepoFormat

- The hg --debug fclone ... option is a good way to see if the forest
  capabilities have been picked up, it will should show you something
  like:

  sending capabilities command
  capabilities: unbundle=HG10GZ,HG10BZ,HG10UN lookup forests changegroupsubset
  sending forests command

  If it doesn't work then the client will print:
  abort: Remote forests cannot be cloned because the other repository 
  doesn't support the forest extension.

Cheers,

Mark