NetworklessInstallationFixes

Differences between revisions 1 and 6 (spanning 5 versions)
⇤ ← Revision 1 as of 2007-10-24 12:02:57 →
Size: 2608
Editor: p54A671ED
Comment:
← Revision 6 as of 2007-11-22 12:52:28 → ⇥
Size: 4618
Editor: 82-69-40-219
Comment: minor tweaks
Deletions are marked like this. Additions are marked like this.
Line 6: Line 6:
 * '''Packages affected''': apt, ubiquity  * '''Packages affected''': apt, apt-setup, ubiquity
Line 10: Line 10:
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec. A networkless installation in ubiquity is currently not as smooth as
it could be. One of the issues is the apt network timeout.
Line 14: Line 15:
This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.		 The package management system is now more robust against unusual
network setups and will no longer take a long time to timeout when
a network connection to the package repository server cannot be
established during install.
Line 20: Line 22:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. Installing without network is very common and should be as painless as possible. Nevertheless, when a network is available, the installer should take the opportunity to install language packs that are not available on the local installation medium.
Line 24: Line 26:
== Assumptions ==  1. Bob installs Ubuntu in a language not available on the CD without a network connected. The installer pauses briefly while it tries to download language packs, but continues without undue delay.
Line 26: Line 28:
== Current status in libapt ==

The current implementation in libapt will set the acquire state to
pkgAcquire::Item::Stat``Transient``Network``Error if it encounters a
"Timeout", "Tmp``Resolve``Failure" or "Connection``Refused" error.

On a Transient``Network``Error, libapt will stop trying to download items
for any sources.list deb/deb-src pair after trying to download the
Release.gpg file. The connection timeout can be controlled via
"Acquire::http::Timeout".

On a system with no network at all the resolver will exit quickly with
"Resolve``Failure". On a system with working DNS but blocked access to
the archive the Acquire::http::Timeout will be run for each
deb/deb-src pair.

That is currently 9 times because of the way the sources.list is
written (lines for "main restricted","universe","multiverse" for each
archive,security,gutsy-updates). It could be collapsed to 3 times if
the sources.list would contain one "main restricted universe
multiverse" line for each "archive, security, updates".

One problem currently is that translations are queued for download as
well and timeout. This should be fixed for hardy. A workaround is to
set APT::Acquire::Translation=none.
 
Line 28: Line 56:
You can have subsections that better describe specific parts of the issue. libapt should be fixed to remember previous network failures on a
given host (in the same apt-get call) and give up immediately.
Line 30: Line 59:
== Implementation == apt-setup currently feeds a single source line to apt for each line it
wants to add to the sources.list. Instead, if it is not going to
comment out lines on failure, it should pass the entire output of each
generator to apt in a single chunk.
Line 32: Line 64:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: This means, that in the worst case we timeout on 3 network sources
(archive.ubuntu.com, security.ubuntu.com, archive.canonical.com). With
a timeout of 10s per source this is 30s. To mitigate this further,
apt-setup should use a cancellable debconf progress bar while running
apt-get update (and map Cancel to SIGINT), in order that a user can
explicitly cancel an update which will never complete.
Line 34: Line 71:
=== UI Changes === ubiquity should fetch the network proxy from gconf immediately before
running apt-setup, rather than relying on gksu to have passed it
(which requires the user to have set the proxy before starting
ubiquity). In addition, since in Hardy we will default to running
ubiquity standalone, we will add an option to ubiquity's Advanced
dialog to set the HTTP proxy.
Line 36: Line 78:
Should cover changes required to the UI, or specific UI that is required to implement this

=== Code Changes ===

Code changes should include an overview of what needs to change, and in some cases even the specific details.

=== Migration ===

Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.		 Finally, we should return to ensuring that sources.list lines are
never commented out on failure, which has been the intention in Ubuntu
installations for some time but stymied by the lack of implementation
of this specification.
Line 51: Line 85:
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during CD testing, and to show off after release. The following network problems can occur:
 * no network at all
 * no working DNS
 * DNS but no connection to the outside world
 * firewall rules that reject packets
 * firewall rules that drop packets
Line 53: Line 92:
This need not be added or completed until the specification is nearing beta. The first two (the most common cases) and the last case should result
in really fast timeouts with the current code already (this needs to
be verified). The remaining cases are problematic and need to be
fixed.
Line 55: Line 97:
== Outstanding Issues ==

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

== BoF agenda and discussion ==

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.		 We need to set up a test framework so that we can monitor how well we
do (possibly use a VM/emulator). Both ubiquity and d-i should be
tested for correct behavior. The SIGINT handler in apt also needs to
be tested; all the tests need to go into the apt regression test
suite.

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Summary

A networkless installation in ubiquity is currently not as smooth as it could be. One of the issues is the apt network timeout.

Release Note

The package management system is now more robust against unusual network setups and will no longer take a long time to timeout when a network connection to the package repository server cannot be established during install.

Rationale

Installing without network is very common and should be as painless as possible. Nevertheless, when a network is available, the installer should take the opportunity to install language packs that are not available on the local installation medium.

Use Cases

  1. Bob installs Ubuntu in a language not available on the CD without a network connected. The installer pauses briefly while it tries to download language packs, but continues without undue delay.

Current status in libapt

The current implementation in libapt will set the acquire state to pkgAcquire::Item::StatTransientNetworkError if it encounters a "Timeout", "TmpResolveFailure" or "ConnectionRefused" error.

On a TransientNetworkError, libapt will stop trying to download items for any sources.list deb/deb-src pair after trying to download the Release.gpg file. The connection timeout can be controlled via "Acquire::http::Timeout".

On a system with no network at all the resolver will exit quickly with "ResolveFailure". On a system with working DNS but blocked access to the archive the Acquire::http::Timeout will be run for each deb/deb-src pair.

That is currently 9 times because of the way the sources.list is written (lines for "main restricted","universe","multiverse" for each archive,security,gutsy-updates). It could be collapsed to 3 times if the sources.list would contain one "main restricted universe multiverse" line for each "archive, security, updates".

One problem currently is that translations are queued for download as well and timeout. This should be fixed for hardy. A workaround is to set APT::Acquire::Translation=none.

Design

libapt should be fixed to remember previous network failures on a given host (in the same apt-get call) and give up immediately.

apt-setup currently feeds a single source line to apt for each line it wants to add to the sources.list. Instead, if it is not going to comment out lines on failure, it should pass the entire output of each generator to apt in a single chunk.

This means, that in the worst case we timeout on 3 network sources (archive.ubuntu.com, security.ubuntu.com, archive.canonical.com). With a timeout of 10s per source this is 30s. To mitigate this further, apt-setup should use a cancellable debconf progress bar while running apt-get update (and map Cancel to SIGINT), in order that a user can explicitly cancel an update which will never complete.

ubiquity should fetch the network proxy from gconf immediately before running apt-setup, rather than relying on gksu to have passed it (which requires the user to have set the proxy before starting ubiquity). In addition, since in Hardy we will default to running ubiquity standalone, we will add an option to ubiquity's Advanced dialog to set the HTTP proxy.

Finally, we should return to ensuring that sources.list lines are never commented out on failure, which has been the intention in Ubuntu installations for some time but stymied by the lack of implementation of this specification.

Test/Demo Plan

The following network problems can occur:

  • no network at all
  • no working DNS
  • DNS but no connection to the outside world
  • firewall rules that reject packets
  • firewall rules that drop packets

The first two (the most common cases) and the last case should result in really fast timeouts with the current code already (this needs to be verified). The remaining cases are problematic and need to be fixed.

We need to set up a test framework so that we can monitor how well we do (possibly use a VM/emulator). Both ubiquity and d-i should be tested for correct behavior. The SIGINT handler in apt also needs to be tested; all the tests need to go into the apt regression test suite.

CategorySpec

NetworklessInstallationFixes (last edited 2008-08-06 16:41:32 by localhost)