Auto-update of native application bundles
tobi at ultramixer.com
Tue Jun 19 00:06:48 PDT 2012
From Apples point of view it's only allowed to save any app specific preferences in ~/Library/Application Support/<ApplicationName>
Am 18.06.2012 um 00:31 schrieb Igor Nekrestyanov <igor.nekrestyanov at oracle.com>:
>> * In your proposal the "app-cast" system is used, where each update
>> is released as a patch (i.e. the 'diff') to get from one version
>> to the next. If an app was out in the wild running at version 2,
>> and the latest release was actually version 5, then to upgrade we
>> would download the v2->v3 patch, install it, then the v3->v4 patch
>> install it, and then the v4->v5 patch and install that (correct me
>> if I got this wrong).
> Sparkle appcast is simply update descriptor (kind of RSS) wth news about update availability.
> It is unrelated to the way update is shipped.
> Moreover but default update is full image of application and can be installed on top of any version.
> Developer may choose to have delta updates if he wants to optimize but he is not required too.
> We are using Sparkle to AU JRE on Mac and it works fine.
>> * In my POC I've gone for the "app-profile" approach, which
>> basically defines the latest release of the system in it's
>> entirety. It's up to each app out there to determine what has
>> changed between it's version and the latest version and download
>> what it needs (i.e. the 'diff'). To go from version 2 to version
>> 5, the app would compare the version of the JARs in its own
>> app-profile against the new one, and download the ones that have
>> changed. There is no download of the 'in-between' versions of 3
>> and 4.
> Sparkle approach does not assume this either.
> Update description must have full image. If there is (optional) delta image it is used but otherwise full image is used.
> IMHO, the biggest differences between appcast xml and app-profile.xml are:
> * appcast is strictly update descriptor and app-profile.xml is application descriptor + versioning info
> * app-profile.xml corresponds to subset of "item" element in the appcast
> * appcast xml may have multiple items - e.g. define application updates for different platforms or several apps
> * item in the appcast describes UPDATE, not application. Application is kind of blackbox and this is good.
> I.e. no knowledge of classpath or specific directory structure. Only what driver knows how to apply update.
> * item has update specific information, e.g. may include release notes that can be used to justify why to upgrade.
> It does not mean we can not use efficient compression (e.g. pack200 for jars) or update parts of app package (e.g. leave alone Java runtime).
> This is up for driver implementation.
> And another important difference is simplier model in Sparkle:
> * ONE version to check (whats available vs update version in appcast)
> [no need to check every jar, resource file, etc]
> * ONE file with update to sign and stage (less hassle for developer)
> [instead of every jar, fewer chance to have staging error (not all files are uploaded, etc)]
> * ONE file to download
> [less chance to have mixed state, e.g. developer may decide to restage/resign app with same version but some jars are already loaded]
> JNLP model is ok but it tries to solve "remote launch" problem and here we are solving other task.
> IMHO, if we separate launch descriptor and update descriptor we will have better model for this problem.
>> It was with this type of usage in mind that I ended up leaning towards the app-profile approach. I imagined my initial upload to my server would be all the JARs for my app, and then when I release subsequent versions I only upload the couple of JARs that have changed and upload the new app-profile.xml (and the build tools could potentially do the upload as well and they would know which jars had changed).
> Essentially you are implementing specific kind of "delta update".
> Except you can dynamically define what delta is needed for arbitrary initial state.
> In sparkle model specifics of delta update are up to the update driver that knows what to do with update package.
> We can support update packages that only update "important" subset of jars, etc.
> For background type of updates size is often not an issue at all (although there are scenarios where it needs to be optimized).
> Anyway, with "update driver" plugins we can extend set of delta updates supported as needed.
>> The flip side of this is that you couldn't choose to go to an in-between version if you wanted to, but I'm not sure that's an overly valid use case?
> Agree, this is not important use case.
> But you may have "different" latest versions - one for XP and another for Vista and another for Mac.
>> With this approach we also have the option of providing a "repair" function. So a client could run a utility to just completely download the entire app and get fresh copies of all the JARs. This is not critical but it could be quite useful. With the app-cast approach, you have to uninstall and then reinstall with your original MSI and then patch (or install with the new MSI) - the uninstall may delete your 'appdata' though, whereas a "repair" could leave this intact.
> IMHO, repair by reinstalling still needs to be supported and existing mechanisms (like MSI or rpm) have means to do this.
> It does not seem as critical requirements for AU module.
>> One other niggling thing for me is how to do things like system properties or VM settings (like -Xmx) without an app-profile. Currently you would be hard coding those into your 'exe' I imagine? So if an update wanted to change these things it would have to recompile the launcher exe and that would have to be part of the 'patch'?
> Bundled app has application descriptor/config file. It is called package.cfg and very simple now.
> It may and need to be improved to pass in things like JVM options.
> In fact launcher is fixed, it is not compiled per app.
> It is embedded into ant task and simply renamed when application bundle is generated.
> All variable parameters are coming from package.cfg
>> There'd be no easy (possible?) way for the user to change these settings on their machine, not sure if that's a good thing or a bad thing though?
> Well, if we need it (it is questionable) then user may tweak package.cfg directly (imho, bad idea. especially if it is system wide) or we could have
> some abstraction of "config" API and application may update its settings per user. But this is unrelated to AU.
> User settings should be left intact after AU. Default app settings are part of application descriptor. So i think we are good with this.
>> And finally one other edge use case I thought of today: it's not uncommon to have several 'apps' included in the one install. For example you might have a user UI and an admin UI. In Java-land, these would ideally share the same libraries but have different 'main' classes. I was thinking the app-profile approach could potentially be extended to cover this but it needs more thinking about, and may just be an unsupported use case. Not sure how/if the app-cast approach could handle this if we said it was a requirement.
> It does not seem hard to me.
> Key question here -- are they shipped as separate installs or one bundle with multiple launchers.
> If they are separate installs they are updated separately and there should be application specific means to ensure they are consistent
> (because user may simply install incompatible versions to start with). Application specific solution may use AU APIs to force autoupdate
> if user tries to launch app that is compatible with others.
> If they are shipped as one bundle then nothing special to be done by AU. AU will update folder with multiple launchers easily as they are just files for AU module.
> It is different story how to bundle msi/exe/dmg for this scenario but this is separate from AU discussion,
>> On Sun, Jun 17, 2012 at 5:26 PM, Igor Nekrestyanov <igor.nekrestyanov at oracle.com <mailto:igor.nekrestyanov at oracle.com>> wrote:
>> Hi Dan,
>> thanks for the great writeup (and pushing this ahead).
>> I finally started to look into this more actively. Some
>> questions/suggestions below.
>> On 6/15/12 5:02 PM, Daniel Zwolenski wrote:
>> *The General Approach*
>> Richard made reference to
>> Sparkle<http://sparkle.andymatuschak.org/> and
>> like him, I feel this is a nice reference point for the style
>> of solution
>> we want. The neat thing that Sparkle does is it provides an
>> API that
>> developers can use inside their code to completely control and
>> the upgrade of their application. This is the approach I've
>> gone for as
>> well. My POC includes a Java API that you can call to check
>> for updates and
>> trigger downloads.
>> IMHO, main great thing about Sparkle is that it is easy to add
>> basic AU functionality to the application
>> (see https://github.com/andymatuschak/Sparkle/wiki)
>> You do not need to use API unless you want custom solution.
>> Therefore i am not focusing here on the public API part (keeping
>> this for later).
>> I think this approach is neat and powerful, but it's worth
>> mentioning that
>> there is still some debate amongst the JFX team about whether
>> to go down
>> this road or avoid the Java API and use off-the-shelf native
>> updaters. Your
>> input will very likely influence this decision so if you have
>> a preference
>> one way or the other (or have other suggestions), be vocal.
>> I think there is no debate on that having some "all in java with
>> minimal native code solution" is good idea.
>> Debate is about whether it should be the only option or it also
>> make sense to support other native update mechanisms
>> (e.g. use Sparkle directly on Mac).
>> As Dan said, thoughts on this are very welcome.
>> An XML deployment descriptor, called 'app-profile.xml' is used
>> to describe
>> the JARs, JRE versions and other launch details - it is
>> comparable to a traditional JNLP file. When an update is
>> triggered via the
>> API, the code checks the remote URL for an updated
>> app-profile.xml. If it
>> finds one it downloads this and checks this with its local
>> Any jars that have different version numbers are downloaded,
>> and if a new
>> JRE version is defined in the app-profile then the zip for this is
>> downloaded and extracted as well.
>> Where these files are placed after download?
>> Once all the downloading is complete, the
>> local app-profile is then overwritten with the newer one and
>> this newer one
>> will be used by the native launcher to set the classpath next
>> time the app
>> is started.
>> IMHO, we can simplify this model following Sparkle ideas.
>> Sparkle does not know anything about application structure, etc.
>> and focus only on update process.
>> This makes it easy to use, simple and robust as it focuses on
>> solves one problem.
>> Important points are:
>> A. Sparkle uses "appcast" to describe what updates are available
>> Appcast includes
>> - version of updates available
>> - meta info about each update (type of update (e.g.
>> regular or mandatory security), release notes, system requirements
>> (e.g. Windows version 7+), etc.)
>> - set of available update packages (aka enclosures) (e.g.
>> full app image, delta patches for several versions, etc.)
>> B. Sparkle knows where to look for update meta info in the
>> application package (on Mac it is part of Info.plist that is some
>> kind of application manifest).
>> However, it only needs to know:
>> - URL of appcast
>> - location of key to verify update signatures
>> - application update settings/preferences
>> The expectation is that once app is updated this info will
>> be updated (as it is part of "update package")
>> [Note: this is not so simple on Windows if we want to make
>> sure we are good citizens and need to update registry]
>> C. Update package is black box for Sparkle core.
>> It is modeled as
>> - file hosted at given URL
>> - signed with given key (signature is part of appcast)
>> - package type defines what "update driver" will be
>> handling this update
>> (this allows to add support for various update
>> packages as plugins/modules)
>> - for most type for packages (except running installers)
>> Sparkle goal is to update set of application files only
>> [Windows or linux require more work as we may need to
>> update registry or package info]
>> D. If admin permissions are needed they only asked before update
>> is finally installed.
>> Sparkle does not keep any internal state in the app folder
>> as it may be only writable for admins.
>> E. Application relaunch after update
>> This is close to your approach but there are some important
>> Following Sparkle more closely may simplify things and at the same
>> time add some desired features
>> (e.g. ability to validate origin of autoupdate)
>> My current thoughts on "ideal" basic experience are as follows.
>> Developer experience (no java coding needed):
>> 1. At the package time developer requests autoupdate support in
>> the native cobundle by providing:
>> * http URL to appcast + public key/cert to validate update
>> * or https url
>> and define application update mode (mandatory update, ask user
>> and allow him to skip version, etc.)
>> 2. Once update is needed developer creates "update package"
>> (single file).
>> For this he will need to run same packager utility/ant task but
>> specify base image, certificate to sign package and
>> and desired type of update (full or incremental)
>> 3. Developer need to create an appcast xml file (default can be
>> generated in step 2 but he may want to customize it)
>> 4. Developer stages both appcast and package bundle
>> End user (background updates):
>> * installs application with background update policy
>> * every time application is launched it will check appcast (if
>> * if update is detected it will be loaded silently while app is
>> * Once update is loaded and verified user will be prompted to
>> install it (it may be notification if update is mandatory)
>> * If he agrees then he may see UAC dialog (unless installation
>> is on per user basis)
>> * application relaunch after update
>> * attempt to launch app being updated should fail with
>> reasonable dialog and should not fail update process (it may need
>> to wait will .exe is unlocked)
>> Implementation wise i was thinking along the lines of what you
>> have (only glimpsed though your code it though) with following
>> notable differences:
>> * update package is single file and we only worry about it version.
>> No need to worry about version of individual components as we
>> do not want arbitrary mix&match.
>> * updates are loaded to ~/.javafx/app-key (where app-key could
>> be derivation from app install location)
>> We also keep user answers for this app in that folder (e.g.
>> had we offered him this version already? when was last time we
>> checked for update, etc)
>> * Update process is:
>> - build temp full app image in the ~/.javafx/app-key/version
>> - if update accepted then exit application and run helper
>> app co copy image over
>> - possibly run "post upgrade script" as part of helper app
>> (this script is specific to "bundler" technology used.
>> E.g. for EXE/MSI bundlers we may want to update few keys in registry
>> - relaunch app
>> - AU module running in the relaunched app could detect temp
>> "image" that is the same as current version and erase it
>> Note that there could be several alternative helper apps.
>> E.g. one that pops UAC and one that does not for per user installs.
>> Or helper up to install update packaged as MSI (by running
>> msiexec silently and showing progress)
>> I was even thinking that "helper" can be implemented in java
>> with minimal native code (to elevate permissions and update
>> registry if needed).
>> * Integration with application bundle
>> - details on whether AU is needed and what type is it may be
>> embedded application manifest for main app jar
>> - embedded jauncher will read them and use internal AU APIs
>> (this is why no user code changes are needed)
>> - current app version, update url and signature are part of
>> package.cfg or manifest in the main app jar
>> - AU support itself is part of jfxrt.jar, bundle also
>> includes helper app to "move image into final location" and
>> script to do "post update config"
>> What do you think?
More information about the openjfx-dev