Ubuntu Wiki

• Launchpad Entry: https://launchpad.net/distros/ubuntu/+spec/make-free-space-wizard

• Created: Date(2006-06-08T17:14:13Z) by SivanGreen

• Contributors: SivanGreen

• Packages affected:

Summary

This specification discusses implemeting a tool , that would suggest to a user several ways to keep his system running undisurbed with a minimum amount of intervention on his side. On the list of ways

Rationale

In due time, a a newly installed system can become cluttered with all sorts of residual content. This can be left over dotfiles that are no longer used cluttering home directories, too many installed kernels, package dependencies that are no longer needed (since their dependant is no longer installed) or log files taking up precious disk space. This sort of content tends to accumulate over time, confusing the user, or even eventually lead to the computer becoming unusable, forcing users to actively put effort into cleaning it up. There should be a solution to warn users before hand, and suggest and carry purge operations for common left over content.

Use cases

• Brian is a Launchpad developer. For developing Launchpad he has several Zope instances installed, and the PostgreSQL data base server running. Those applications produce alot of log file data, especially under heavy development and experimentation, where they are usually set to maximum verbosity for debugging purposes. After a week of heavy work, his free space on his / fs reaches the low minimum. The system clean up tool detects that, and before Brian is running into operation problems, it offers him to free some old logs, and some files under /tmp that occupie most of the space currently on his / fs. He confirms the removal, and resumes working (the wizard takes care to do the removal in the background, first hunting for the biggest files, in order to maintain operational state of the system in the shortest possilbe time). The interaction with Brian is done through the desktop notification infrastructure.
• George has been reciving lots of audio visual content recently, from relatives overseas. He burnt them all to DVDs, and moved to the garbage bin every file he had burnt. After a while of doing so, he does not notice his free space in his home folder dropped to the minimum allowed. He is also downloading a big iso file of the edgy desktop-cd for testing. The system clean up tool detects that there is not enough free space, and pops up a desktop notification for George, suggesting that there is large amount of data in .Trash, that can be purged, to make more room. George acknowledges, space is freed and the download is saved.
• David had an issue after burning a DVD where the drive would not unlock. In his wisdom, he used a paper clip to force the tray to open and retrieved the disk. Little did David know that the kernel.log and messages.log were filling up with information about the bad IDE communication thingo until the system clean up tool detects that the hard drive is filling up and provides a notification that the data can be purged to make more room. David acknowledges, space is freed and the system operates normally.
• Dan has been using ubuntu for some time now, and has been experimenting with the wealth of software packages available from the various ubuntu repositories. He is not aware that many of the package pulled in as dependencies are not removed when he chooses to remove the package he originally installed. When disk space becomes low, the tool checks to see if there are any packages that can be good candidates for removal (orphaned, missing dependencies etc) and asks if he would like to have them removed. He acknowledges, and those package get removed, bringing back the lost disk space.

• High Priority:John is a dapper user. A recent kernel upgrade has been released to cater for a security bug. After rebooting to his new kernel, the system informs him that he has some left over kernel packages that could be removed and asks him if he wants to do so. If he acknowledges, all of the kernels that he no-longer needs are removed, leaving him with a clean /boot with only the currently running kernel and the newly installed one.

Scope

• Kernel left overs:
  1. Due to security upgrades.
  2. General bug fixes and version upgrades.
  3. Make sure never to touch a kernel package created by the user.
     • -- How can we know if it's such a pakcage ??
• Residual packaging related content:
  1. Conffiles,
  2. Init scripts.
  3. Dotfiles that were introduced using a package and their parent package is no longer on the system.
• General letf over content:
  1. Browser caches,
  2. Aged audiovisual content.
  3. Aged and/or large log files.
  4. Orphan dotfiles.
  5. Large ISO files.

Design

Aging: The time period that had passed between the last time a file has been accessed, and the recorded time reference point. The reference point can be the current date and time reported by the system, or an eariler time in order to enable more accurate aging calculation when a system hasn't been used over a long period of time.

1.Dealing with general Left over content:

A weighing algorithm needs to be developed to enable the tool to identify targets of opportunity. The following factors needs to be taken in consideration in producing the weight result per file:

1. A relative time reference point should be used for measuring the aging time of all files. This in order to overcome the "vacation problem" where a user hasn't been using his system for a long period of time, and by using the current time when he first login after his vacation, the weighing would get distorted to include files that the user accessed just before he went on vacation. This means that we need to measure the actual usage time. To do so, we will record the last access time of files that are access every login (for example, gdm files) and use that time as our new reference point.
2. How much time passed since last access time of the file.
3. Which mime / file type.
4. Size of file
5. Capacity of the holding volume or the user set quota.

While calculating its opportunities, this alogrithm should make sure its not being mislead by raw agin

1. The vacation problem. If a user goes on vacation and does not access his computer for a relatively long period of time, then we have to make sure that the wizard doesnt suggest files to be removed that the user "just accessed" before going on vacation. To deal with that, the time measured since the last access operation of a file
2. There should be also an option to configure the tool to never scan for specific directories, to be configured by more power users that would like to have some folders untouched.
3. The tools should ignore specific ~/.?* directories to not break applications in ubuntu-desktop and other special cases.

2.Package left over house keeping: (orphand files, unneeded dependency packages)

1. Offer to remove orphaned files that no longer belong to any of the packages installed on the system.
2. Offer to remove packages that were installed due to satisfying dependencies of other packages that are not longer installed.
3. Offer to remove packages that are rarely or not used anymore.

3.Unused left over kernels:

1. When a new kenrnel is being installed by the GUI packaging tools, the installing packaging tool will call the system clean up tool with a command line that will instruct it to deal only with kernels clean up on that specific invocation:
2. Then, the system clean up tool will mark the packages of:
   1. The currently running kernel.
   2. The new kernel that was just installed.
3. Checking if the user has any other kernels installed other then those in the afformentioned list.
4. If he does not, do nothing.
5. If he does have, gather a list of all those kernel packages.
6. Popup a desktop notification to the user "You have un-used kernels installed on the system. Would you like me to clean them up?".
7. If the user confirms, then present a dialog displaying the list of kernels that was constructed in steps 2-4 , in the a window that will contain a columnd table, each row representing a kernel package:
   • Row 1: Kernel Version. (e.g. "2.6.15-25-686")
   • Row 2: Kernel Package Name. (e.g "linux-image-2.6.15-25-686").

   • Row 3: A check box indicating if this kernel package is to be removed, or left installed. (checked->keep, unchecked->remove)

8. All items in the list are by default unchecked (meaning , tool will remove all kenrels on the remove-list)
9. The user can choose to keep any of the kernls on the list by checking the checkbox next to the name/version.
10. Pressing "Commit" , will calculate the kenrels packages to be removed from the check list, remove the kernel packages, and notify the user of success, or error if any issues were encountered while removing.
11. Left over kenrel removals should be also proposed when a user is running out of sufficient free space in his /boot fs or in his / while /boot is part of it. (this should be the last priority if /boot and / are on the same fs, we should first check for other bigger files that can be removed)

4.Modes of operation:

1. When being executed as an unpriv'd user, the tool should touch running user's home folder only.
2. Executed as a priv'd user, (sudo'd) tool should care about system wide cleanup (kernel, system folders, etc)

5.Catching disk space events:

1. The tools should either use gnome-volume-manager to catch for low disk space events being dispatched.
2. If it cannot be customized to allow different criteria for this events to dispatch (by quota, user selected minimum free space) then the tool should probably have a deamon of it's own to do that. This would be suboptimal since it will mean another python interpreter running and another daemon consuming system resources.

Implementation

• PackageDependencyManagement should be used as much as possible to achive design item #2.

Code

Data preservation and migration

Outstanding issues

BoF agenda and discussion

* It would be better if the computer can handle this problem by itself. Certainly there should be an alert warning you that a disk is nearly full. But sources of ever-expanding files are bugs that should be fixed at the source: for example, [http://bugzilla.gnome.org/show_bug.cgi?id=149572 Gnome bug 149572]. Another very cool way to help people free space on a disk would be a folder view for Nautilus that does the same thing as [http://xdiskusage.sourceforge.net/ xdiskusage] or [http://derlien.com/ Disk Inventory X]. -- MatthewPaulthomas

* May be useful to have an enterprise mode that assists users in cleaning out their home directory if they meet their quota. Would entail adding quota support. (may need this anyhow?) Also limiting the tool to specific directories may be useful. -- ScottDier

* When running as a unpriv'd user, tool should only allow user to clean and tidy his own home folder, possibly use two different front ends per priv'd and unprived operation.

* Use gnome volume manager to catch notifcations about low disk space. Research should be put into evaluating how customizable gnome-volume-manager is for the purpose. (if we need to maybe check quotas, precentage and be able to have different minimum free space values per different file systems)

* Have a weighting to order files to display for possible deletion that uses space, last access time, and filetype detected by libmagic. (ie: large and old core files may be listed has highly probable for deletion)

* Only show ~/.?* directories as a summary unless user specifically asks. When a user wants to investigate these files further then warn user that removing such files may break applications or cause data loss and should only be used by experienced users.

* For ~/.?* directories only summarise them in the list if all contents are not used for long period of time.

* Ignore specific ~/.?* directories to not break applications in ubuntu-desktop and special cases.

* Write a research script , that will be executed on users machines , to find out:

• How much time passed since last access time of the file.
• Which mime / file type.
• Size of file
• Weigh those files , present it to the user and ask for the opinion for how good the weighing is, is the right files are on the list? maybe some files can grow big and never touch them, etc.
• Have feedback system integrated into the application. For example a checkbox to send anonymized data. This data might incluse a-f-wizard usage patterns and calculated weights.

• vacation problem -> if a user goes on vacation and gets back from it then we have to make sure that the wizard doesnt suggest files to be removed that the user "just accessed" before going on vacation. So actual computer use is important for the weighting.

• human readable weighting file. Some simple scripting language. Users can adjust the weighting algorithm themselve and give feedback on things that work..... eventually templates.

* Allow a user to bring up nautilus in order to view or work out a file that has been on the "under-the-gun" list.

CategorySpec