AutomaticUpgrade
|
Size: 11418
Comment: udu
|
Size: 6263
Comment: adressing scotts comments
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from UbuntuDevel/PackageDependencyManagement = Package Dependency Management = |
|
| Line 3: | Line 7: |
| * '''Launchpad entry:''' https://launchpad.net/distros/ubuntu/+spec/release-upgrades * '''Created:''' [[Date(2005-10-06T19:41:30Z)]] by JaneWeideman * '''Contributors:''' * '''Packages affected:''' `update-manager` |
* '''Launchpad Entry''': https://launchpad.net/distros/ubuntu/+spec/dependency-removal * '''Created''': [[Date(2005-11-01T18:24:04Z)]] by MichaelVogt * '''Contributors''': MichaelVogt, DanielBurrows * '''Packages affected''': apt, aptitude |
| Line 8: | Line 12: |
| == Summary == | == Introduction == |
| Line 10: | Line 14: |
| This is a proposal for an application capable of easing the transition from one version of Ubuntu to the next. | This specification describes an enhancement to the information the system stores about packages that are installed locally. The goal is to improve the ability of systems to evolve over time to continue to reflect the current Ubuntu release. These primary improvements are envisaged: 1. The system needs to have some idea '''why''' a package was installed. Some of the higher-level package management tools, such as `Aptitude`, already do something similar to this. They track, for example, whether or not a package was installed because of a specific request by the system administrator, or to satisfy the dependencies of a package that was being installed. This allows them to offer to remove those dependency packages when the chosen package is removed later. 1. Over time, it is inevitable that the Ubuntu team may choose to change the open source products preferred for specific functionality in an Ubuntu system. For example, the Ubuntu developers might choose to migrate from `Postfix` to `Nullmailer`, or from `Esound` to `Polyp``Audio`. For new installations that presents no problem, someone installing the new release will simply get the new apps. However, for someone upgrading, it will be necessary to identify whether or not the user has modified any configuration files for the relevant app, and if not, to migrate the system to the new app. |
| Line 14: | Line 27: |
| At the moment people who want to upgrade from one version of Ubuntu to the new release have to update their sources.list manually, and then do a dist-upgrade. For most people not experienced with Ubuntu, using the command-line is a big step (''see'' CommandLineDisintegration), especially where it concerns something as important as an upgrade. Aside from this, many people may not know that a new version of Ubuntu is available. | Over time, a Linux system accumulates a substantial amount of cruft. Traditionally, installing a package requires installing its dependencies. Later, when that package is upgraded, new dependencies might be introduced and older ones dropped, but in most systems the old package dependencies are never uninstalled. The end result is a plethora of unwanted and un-needed libraries and supporting packages which take up disk space and potentially also slow down the day to day operation of the computer. |
| Line 16: | Line 35: |
| The Starter Guide in Ubuntu 5.10 contains instructions on how to upgrade ... to Ubuntu 5.10. Instructions on how to upgrade to Ubuntu 6.04ish could not be included because they depended on knowing the codename for that release, and the codename was not announced in time. | These improvements will make it easier to keep Ubuntu systems lean-and-mean over a series of releases, leaving only those packages which the Ubuntu team recommends, the packages explicitly selected by users, and the necessary dependencies required to support those packages. |
| Line 18: | Line 40: |
| One of the difficulties of actually doing an upgrade is being confronted with unexpected questions about how to deal with config files during the upgrade itself. These come at a bad time, since the user can't use the machine that is being upgraded to search the web for help. So it would also help if the system could simulate a "dry run" of the upgrade or look for changed configuration packages or somehow provide upgrade information tailored to the set of packages currently installed. |
== Scope and Use Cases == |
| Line 23: | Line 42: |
| == Use cases == | The following use cases illustrate the ideas in this spec: |
| Line 25: | Line 44: |
| Fred is running Breezy. He is using a fast Internet connection. When Dapper is released, his Update Manager notices that there's a new release available and that his system will have to be rebooted. Fred accepts the upgrade at the end of his work day, goes home, and his machine is running Dapper when he comes back next morning. | 1. Jimmy installed Breezy, and has now updated to the next release. During the update, no matter which package management tool he uses, old dependencies will be removed and new ones installed. There should be no old libraries left on his system once the upgrade is complete. |
| Line 27: | Line 46: |
| Jane does not have an Internet connection, but she got a new Dapper CD-ROM from Shipit. She puts the disk in her drive, and the same thing happens. | 1. Annabel has installed openldap, and all of its dependencies were automatically installed on her system. Now she is going to remove it from this system using a different package manager. That package manager identifies packages that were installed purely in support of openldap, and offers to remove those too. |
| Line 29: | Line 48: |
| Joe has also installed Breezy by CD-ROM. He does have a low-bandwidth Internet connection. When Dapper is released, his system notices and asks him whether he wants to update. He declines, because his Internet connection is too slow. After a month, he too gets a new Dapper CD-ROM and upgrades his system just like Jane did. * s/Dapper/Dapper+1/ s/Breezy/Dapper/ ? == Scope == We are concerned with helping the user update from one release of a distribution to the next one, via either the network or a CD/DVD-ROM with .deb files. Any other update, such as from Debian to Ubuntu, from $release to $release+2, or via the live / UbuntuExpress CD (which does not have .deb files), is out of scope. |
1. Jack is upgrading from Breezy to the next Ubuntu release. His system has postfix installed, because that was the default mail server in Breezy, but he has never modified its configuration and so when he does the update, postfix is removed and replaced with newmailer, the Ubuntu team's chosen replacement for postfix in the new release. |
| Line 43: | Line 52: |
| There are three steps that needs to be taken: 1) The user must be informed about the new available distro (possibly release notes as well) and run the upgrade tool 2) The upgrade tool must be able to download updated information how to perform the upgrade (e.g. additional steps like upgrading certain libs first) 3) The upgrade tools runs and installs/remove packages and does some additional steps like post-release cleanup |
For the use case 1 we are going to mark each package `automatic` if it was installed only to satisfy a dependency. If a package is removed later a `mark-and-sweep` algorithm is run that marks any package that is part of a dependency of a given root-set (essential+manually installed). Any package that is notm marked here has no reverse dependencies and can be safely removed. |
| Line 52: | Line 54: |
| === Inform the user === | The use case 2 is adressed by the upgrade tool discussed at https://wiki.ubuntu.com/AutomaticUpgrade. It will identify changes in meta-packages based on the information we provide and suggest the removal of the superseded package. |
| Line 54: | Line 56: |
| The user is informed about new releases with the meta-release file that is downloaded from the net. It contains information like codename, version number and date of the release. The release file fetching is part of the update-manager application. When a new distro release is found it adds some button to the gui that allows a dist-upgrade. | All packages installed by the installer are installed via aptitudes task matcher right now. That means that the auto-flag is set for a lot of the libraries but not for the installed applications. That means that removing e.g. ubuntu-desktop will *not* result in the suggestion to automatically remove all it's dependencies. |
| Line 56: | Line 58: |
| === Get the upgrade information === | == Implementation Plan == |
| Line 58: | Line 60: |
| A upgrade may involves more steps than to upgrade the packages. It may be required to upgrade certain packages first (like libgtk2.0-0 for the hoary->breezy upgrade because of a bug that made gtk apps crash) or do other stuff that may involve changing files or other scary stuff. | For the use cases Number 1 the automark feature of aptitude needs to be ported to libapt. It should be easily accessible to all the frontends (apt-get.cc, synaptic, aptitude, python-apt). |
| Line 60: | Line 62: |
| Because we do not know the exact steps at the time the tools is written this upgrade information must be totally flexible. It also must not be part of the regular packaging system because the user may not have run "apt-get update" for a long time and therefore apt does not know about the existance of the package. | === Data Preservation and Migration === |
| Line 62: | Line 64: |
| The information must not be in a package included in the distribution because we want to be able to solve new problems as they are discovered. | The current flags in aptitude will be converted automatically on the first run of aptitude. |
| Line 64: | Line 66: |
| A mechanism is proposed that involves downloading a gpg signed tar.gz from a URL. That tar.gz will contain a executable that is run and may contain additional stuff that may be required (even packages that needs to be installed). The executable "is" the upgrade information. It will do all the needed steps automatically that are usually written in the release notes. The update tool downloads the tarball, unpacks it, runs the script and make use of possible additonal stuff inside the tarball. Any actions that are taken need to communicate with a frontend that displays a nice and user-friendly screen. |
=== Packages Affected === |
| Line 67: | Line 68: |
| The mechanism needs to be flexible enough to support cd upgrades as well without any network sources. If the network is available, it should still check there for a new version of the upgrade tarball (embarrassing bus may still happen...) |
The `apt` package needs to be improved to provide support for the marking of automatic dependencies. This feature needs to be exported so that frontends like synaptic, python-apt can use it too. |
| Line 70: | Line 70: |
| === The actual Upgrade === | === User Interface Requirements === |
| Line 72: | Line 72: |
| Before the actual upgrade, some sanity checks needs to be performed. For instance, checking that the cache has a BrokenCount == 0 or that no unusal amount of 3rd party packages is installed. | A new "apt auto-remove" commands will be added that suggests to remove the automatically installed packages. Synaptic needs to be updated to use this feature too. Python-apt needs to support it too. Aptitude should make use of the feature as well (use the version in libapt instead of the version in aptitude). |
| Line 74: | Line 74: |
| The steps that needs to be taken in order to guarantee a upgrade are run next. That includes things like installing/upgrading packages before the actual dist-upgrade, modify sources.list (with a /etc/apt/sources.list.d/ entry) or run random scripts. Because a upgrade always involves removing packages there must be some sanity checks to ensure that nothing is removed that shouldn't be. Checking if e.g. no Kernel, X, or meta-packages are removed needs to be checked. For meta-packages a additional flag (MetaPackage) should be used in the Packages file. Additionaly it should be possible to have a whitelist in the upgrade-package that knows about save removals. We should remove packages that are obsolete and were available in the old release only (or at least display a list of those packages). That requires to save the Origin of the installed packages before the upgrade. We should having different GUIs for the upgrade tool (console,gtk). People should be able to get information about the new distro even when they don't run X. These goals (provide a server upgrade) may not make it into the first version and that will probably be superseded by smart because it contains a "abstract" gui already. == Implementation == === Step 2, getting the upgrade information === We should publish it through the launchpad archive infrastructure and use the ubuntu-archive key for it. The tarball should be called dist-upgrade.tar.gz and should be in dist/$distro release context and it should contain a executable with the same name. It is uncompressed into a temp directory. See the CustomUploadFormats spec also. If any packages are contained in the tarball we must include the source (because of the GPL). It also means that it would be arch-specific (something we dont't want to do). Daniel Silverstone needs a Python function which installs the tarball into the dists/ tree, adds any relevant symlinks, etc., The changes file needs to be in section "raw-dist-upgrade". We should also add upload validation (another Python callable which checks "is this a tarball, does it install all of its files in the right place, does it include an upgrade script, ..."). === Step 3, install the upgrade === An extension to UpdateManager written in python-gtk/python-apt (with a nice abstraction for the frontend) that is able to perform dist-upgrades. It should be able to To find cruft we have to generate a list of obsolete packages: 1) unknown origin (dpkg -i, whatever) 2) obsolete (in new release) 3) obsolete obselete (eg obsoleted in warty -> breezy upgrades) we can do this by pre-calculating what obsoletes we had in the previous upgrades and put it in the upgrade script Check for space constraints. Preservation of user package selection (e.g., via metapackages) -> daniels new problem-resolver into python-apt / libapt and make sure that the problemresolver never removes them Warnings about unsupported packages? -> if anything from universe is installed warn the user that the upgrade may not be as smooth as it should be Do something sane with old kernel(s) -> trying to keep not too many kernels around. |
Applications are usually save from being suggested to automatically removed because they are usually in the top of the depedency chain and will not be pulled in as a part of a dependecny. Aptitude will work as before, synaptic will present packages that can be removed automatically but it will not mark them for removal automatically. |
| Line 116: | Line 77: |
| === Code === | == Outstanding Issues == |
| Line 118: | Line 79: |
| === Data preservation and migration === | A implementation (that needs testing) is available in the michael.vogt@ubuntu.com--2005/apt--auto-mark--0 branch. Close cooperation with Daniel Burrows (the aptitude author/maintainer) is taking place, he branched from the initial port and added the missing bits that he needed for aptitude. python-apt code is written as well. All the code needs testing. |
| Line 120: | Line 81: |
| == Outstanding issues == == BoF agenda and discussion == == See also == See also the UDU discussion https://wiki.ubuntu.com/SystemUpgradeTool == Discussion before the BOF == My proposed solution is to make a small application that will depend on the desktop metapackages (ubuntu-desktop, kubuntu-desktop, xubuntu-desktop), and will have a menu entry in System->Administration (and its KDE/XFCE equivalents).[[BR]] This application would only become part of the ubuntu-desktop package once the next release has been released, at which time it would be automatically uploaded to everybody who hasn't removed the ubuntu-desktop meta-package.[[BR]] After this had happened, there could be a notification similar to what now happens after you upgrade the kernel, only stating that there is a new version and that the user might want to look at the Automatic Upgrade application.[[BR]] The application itself would only contain several buttons:[[BR]] * [Upgrade to Breezy] - would change the sources.list, and do a dist-upgrade. * [Show me the release notes] - would open the release notes in the user's browser. Preferably a local copy that came with the application and also contains links to where the CDs can be downloaded. * [Where can I download the CD?] - would open up the page with the CDs that can be downloaded for those without the necessary connection, but able to download them elsewhere. This might be overkill as the links are already present in the release notes as well. * [Send me the CD] - would open up the [https://shipit.ubuntu.com/ ShipIt page], or a simple form that already takes the information from About Me. * [Cancel] - for those who don't want to upgrade now. * [Remind me one month from now] - a checkbox for periodic reminders. === Pros === * This solution would let users know that a new version is available, and offer them an easy way to upgrade to this version. * Users who do not want to update only have an unobtrusive entry in their Administration menu, and could in theory remove that. Although doing that through synaptic would break the ubuntu-desktop meta-package. === Cons === * This solution only works for users who still have the (x|k)ubuntu-desktop metapackage installed. On the other hand, the users who no longer have that are most likely the ones who know how and when to do an upgrade anyway. * There would be a high clutter factor for in having an entire Administration menu item for something that, on average, is going to be used less than once in six months (since some people will upgrade using a CD or not at all). |
Package Dependency Management
Launchpad Entry: https://launchpad.net/distros/ubuntu/+spec/dependency-removal
Created: Date(2005-11-01T18:24:04Z) by MichaelVogt
Contributors: MichaelVogt, DanielBurrows
Packages affected: apt, aptitude
Introduction
This specification describes an enhancement to the information the system stores about packages that are installed locally. The goal is to improve the ability of systems to evolve over time to continue to reflect the current Ubuntu release.
These primary improvements are envisaged:
The system needs to have some idea why a package was installed. Some of the higher-level package management tools, such as Aptitude, already do something similar to this. They track, for example, whether or not a package was installed because of a specific request by the system administrator, or to satisfy the dependencies of a package that was being installed. This allows them to offer to remove those dependency packages when the chosen package is removed later.
Over time, it is inevitable that the Ubuntu team may choose to change the open source products preferred for specific functionality in an Ubuntu system. For example, the Ubuntu developers might choose to migrate from Postfix to Nullmailer, or from Esound to PolypAudio. For new installations that presents no problem, someone installing the new release will simply get the new apps. However, for someone upgrading, it will be necessary to identify whether or not the user has modified any configuration files for the relevant app, and if not, to migrate the system to the new app.
Rationale
Over time, a Linux system accumulates a substantial amount of cruft. Traditionally, installing a package requires installing its dependencies. Later, when that package is upgraded, new dependencies might be introduced and older ones dropped, but in most systems the old package dependencies are never uninstalled. The end result is a plethora of unwanted and un-needed libraries and supporting packages which take up disk space and potentially also slow down the day to day operation of the computer.
These improvements will make it easier to keep Ubuntu systems lean-and-mean over a series of releases, leaving only those packages which the Ubuntu team recommends, the packages explicitly selected by users, and the necessary dependencies required to support those packages.
Scope and Use Cases
The following use cases illustrate the ideas in this spec:
- Jimmy installed Breezy, and has now updated to the next release. During the update, no matter which package management tool he uses, old dependencies will be removed and new ones installed. There should be no old libraries left on his system once the upgrade is complete.
- Annabel has installed openldap, and all of its dependencies were automatically installed on her system. Now she is going to remove it from this system using a different package manager. That package manager identifies packages that were installed purely in support of openldap, and offers to remove those too.
- Jack is upgrading from Breezy to the next Ubuntu release. His system has postfix installed, because that was the default mail server in Breezy, but he has never modified its configuration and so when he does the update, postfix is removed and replaced with newmailer, the Ubuntu team's chosen replacement for postfix in the new release.
Design
For the use case 1 we are going to mark each package automatic if it was installed only to satisfy a dependency. If a package is removed later a mark-and-sweep algorithm is run that marks any package that is part of a dependency of a given root-set (essential+manually installed). Any package that is notm marked here has no reverse dependencies and can be safely removed.
The use case 2 is adressed by the upgrade tool discussed at https://wiki.ubuntu.com/AutomaticUpgrade. It will identify changes in meta-packages based on the information we provide and suggest the removal of the superseded package.
All packages installed by the installer are installed via aptitudes task matcher right now. That means that the auto-flag is set for a lot of the libraries but not for the installed applications. That means that removing e.g. ubuntu-desktop will *not* result in the suggestion to automatically remove all it's dependencies.
Implementation Plan
For the use cases Number 1 the automark feature of aptitude needs to be ported to libapt. It should be easily accessible to all the frontends (apt-get.cc, synaptic, aptitude, python-apt).
Data Preservation and Migration
The current flags in aptitude will be converted automatically on the first run of aptitude.
Packages Affected
The apt package needs to be improved to provide support for the marking of automatic dependencies. This feature needs to be exported so that frontends like synaptic, python-apt can use it too.
User Interface Requirements
A new "apt auto-remove" commands will be added that suggests to remove the automatically installed packages. Synaptic needs to be updated to use this feature too. Python-apt needs to support it too. Aptitude should make use of the feature as well (use the version in libapt instead of the version in aptitude).
Applications are usually save from being suggested to automatically removed because they are usually in the top of the depedency chain and will not be pulled in as a part of a dependecny. Aptitude will work as before, synaptic will present packages that can be removed automatically but it will not mark them for removal automatically.
Outstanding Issues
A implementation (that needs testing) is available in the michael.vogt@ubuntu.com--2005/apt--auto-mark--0 branch. Close cooperation with Daniel Burrows (the aptitude author/maintainer) is taking place, he branched from the initial port and added the missing bits that he needed for aptitude. python-apt code is written as well. All the code needs testing.
AutomaticUpgrade (last edited 2008-08-06 16:19:16 by localhost)