DebuggingNetworkManager
|
Size: 7454
Comment:
|
← Revision 57 as of 2023-06-27 20:21:45 ⇥
Size: 5239
Comment: Added brief section on netplan
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Available languages: [[https://wiki.ubuntu.com/DebuggingNetworkManager_it| Italiano]], | |
| Line 3: | Line 4: |
| == Bug Summary == | = Bug Summary = |
| Line 11: | Line 13: |
| == Understanding your bug and getting more information == | = Understanding your bug and getting more information = |
| Line 13: | Line 15: |
| * There is a lot of debugging information available on the GNOME Live wiki: [[http://live.gnome.org/NetworkManager/Debugging|NetworkManager/Debugging]]. | * There is a lot of debugging information available on the GNOME Live wiki: [[https://wiki.gnome.org/Projects/NetworkManager/Debugging|NetworkManager/Debugging]]. |
| Line 17: | Line 19: |
| == Getting debug logs == | = Getting debug logs = |
| Line 19: | Line 21: |
| First, make sure you have the debug helper script: [[http://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/test/debug-helper.py]]. | You can then follow developers' intructions on a bug report for the exact command line to use; or run it directly as such. |
| Line 21: | Line 23: |
| You can then follow developers' intructions on a bug report for the exact command line to use; or run it directly as such: | == Getting NetworkManager debug logs == |
| Line 23: | Line 25: |
| === Getting NetworkManager debug logs === | By default, the NetworkManager log level is set to info. You can use nmcli to modify the logging level: |
| Line 26: | Line 28: |
| sudo python debug-helper.py --nm debug | $ sudo nmcli general logging level DEBUG domains ALL |
| Line 29: | Line 31: |
| Then get the logs which will be written to /var/log/syslog. To disable it, pass '''info''' instead of '''debug''' in the above command, or reboot. === Getting ModemManager debug logs === |
You do not need to restart NetworkManager to begin seeing debug messages logged to journalctl. You can watch the NetworkManager logs: |
| Line 34: | Line 34: |
| sudo python debug-helper.py --mm debug | $ sudo journalctl -f -u NetworkManager |
| Line 37: | Line 37: |
| Then get the logs which will be written to /var/log/syslog. To disable it, pass '''info''' instead of '''debug''' in the above command, or reboot. | == Getting ModemManager debug logs == |
| Line 39: | Line 39: |
| See also [[DebuggingModemmanager]]. === Getting wpasupplicant debug logs === |
Manually run with debug enabled: |
| Line 44: | Line 42: |
| sudo python debug-helper.py --wpa debug | $ sudo /usr/sbin/ModemManager --debug $ sudo /usr/sbin/NetworkManager --debug --log-level=DEBUG |
| Line 47: | Line 46: |
| Then get the logs which will be written to /var/log/syslog. To disable it, pass '''info''' instead of '''debug''' in the above command, or reboot. | See also [[https://modemmanager.org/docs/modemmanager/debugging/|DebuggingModemmanager]]. |
| Line 49: | Line 48: |
| === Getting a capture of syslog === | You do not need to restart ModemManager to begin seeing debug messages logged to journalctl. You can watch the ModemManager logs: {{{ $ sudo journalctl -f -u ModemManager }}} == Getting wpasupplicant debug logs == Change the log level: {{{ $ sudo wpa_cli log_level debug }}} You do not need to restart wpa_supplicant to begin seeing debug messages logged to journalctl. You can watch the wpa_supplicant logs: {{{ $ sudo journalctl -f -u wpa_supplicant }}} == Getting a capture of syslog == |
| Line 57: | Line 75: |
| tail -n0 -f /var/log/syslog > /tmp/syslog | $ tail -n0 -f /var/log/syslog > /tmp/syslog |
| Line 72: | Line 90: |
| === Handling 3G / modem issues === | == Handling 3G / modem issues == |
| Line 74: | Line 92: |
| An few extra things that are very helpful to add in case of issues with 3G: | Here are a few extra things that are very helpful to add in case of issues with 3G. |
| Line 86: | Line 104: |
| == A Testcase == | == Captive portal == You can check the status from the cli using: {{{ $ nmcli networking connectivity check }}} Since 1.38, you can set Environment=NM_LOG_CONCHECK=1 in NetworkManager.service and restart the service to get additional debug logging about connectivity checking. = A Testcase = |
| Line 90: | Line 117: |
| Stop NetworkManager |
First, stop NetworkManager and unload your driver: |
| Line 93: | Line 119: |
| sudo stop network-manager | $ sudo systemctl stop NetworkManager $ sudo modprobe -r DRIVER |
| Line 96: | Line 123: |
| To unload your driver {{{ sudo modprobe -r DRIVER }}}. Then load the driver {{{ sudo modprobe DRIVER }}} and start NetworkManager: |
Next, load the driver and start NetworkManager: |
| Line 101: | Line 125: |
| sudo start network-manager | $ sudo modprobe DRIVER $ sudo systemctl start NetworkManager |
| Line 104: | Line 129: |
| = Netplan = | |
| Line 105: | Line 131: |
| == Debugging Crashes == | A configuration abstraction mechanism has added to NetworkManager as of Ubuntu 23.10, called Netplan. There is a whole [[https://netplan.readthedocs.io/en/stable/|library of netplan documentation]] available. However here is a very basic amount of netplan debugging. |
| Line 107: | Line 133: |
| Netplan uses the files in /etc/netplan/*.yaml to generate the running profiles you find at /run/NetworkManager/system-connections/*.nmconnection. If you suspect a feature defined in your /etc/netplan/*.yaml files is not being used, it is a good idea to check that the content matches up. | |
| Line 108: | Line 135: |
| To install debug symbols, add the following line to your {{{/etc/apt/sources.list}}} {{{ deb http://ddebs.ubuntu.com/ natty main universe }}} or {{{ deb http://ddebs.ubuntu.com/ oneiric main universe }}} Then install the appropriate dbgsym packages: {{{ sudo apt-get update sudo apt-get install network-manager-dbgsym libnm-util2-dbgsym libnm-glib4-dbgsym libglib2.0-0-dbgsym }}} Then stop NetworkManager: {{{ sudo stop network-manager }}} Run NetworkManager in a debugger {{{ sudo gdb /usr/sbin/NetworkManager ... (gdb) run --no-daemon }}} Once it crashes get a backtrace {{{ (gdb) bt ... (gdb) bt full ... (gdb) thread apply all bt full ... }}} and attach the backtrace above together with your {{{/var/log/syslog}}} to the bug. == Bug Tags == These tags allow isolation of bugs into smaller groups, providing an easier and faster way to work on specific issues. ||<rowbgcolor="#FFEBBB"> '''Tag''' || '''Use case''' || || [[https://launchpad.net/ubuntu/+bugs?field.tag=driver-madwifi|`driver-madwifi`]] || Bugs in which the madwifi driver is in use || || [[https://launchpad.net/ubuntu/+bugs?field.tag=driver-ndiswrapper|`driver-ndiswrapper`]] || Bugs in which the ndiswrapper driver is in use || || [[https://launchpad.net/ubuntu/+bugs?field.tag=vpn|`vpn`]] || Bugs related to either openvpn, vpnc or pptp network-manager vpn modules || || [[https://launchpad.net/ubuntu/+bugs?field.tag=encryption-wep|`encryption-wep`]] || Bugs in which WEP encryption is used || || [[https://launchpad.net/ubuntu/+bugs?field.tag=encryption-wpa|`encryption-wpa`]] || Bugs in which WPA encryption is used || || [[https://launchpad.net/ubuntu/+bugs?field.tag=encryption-wpa2|`encryption-wpa2`]] || Bugs in which WPA2 encryption is used || The previously described tags are specific to the [[NetworkManager]] application, if you need more general tags please visit [[Bugs/Tags]] page. == Testing a patch == It's often a good idea to check in the [[https://edge.launchpad.net/~network-manager/+archive/trunk|NM trunk PPA]] to see if an updated version of NetworkManager fixes the bugs you see. Be careful though, as those are daily builds that may break other things, and are to be considered unstable. Sometimes a developer might ask you to test a quick patch he came up with. Below an example how to do that for network-manager. If your bug is for network-manager-applet or some other package, just replace the package name See [[DebuggingModemmanager]] for the same instructions for ModemManager. {{{ # get build dependencies sudo apt-get install build-essential fakeroot devscripts sudo apt-get build-dep network-manager # get sources sudo apt-get source network-manager # cd into the source directory cd network-manager-*/ # apply the patch (note that -p1 could also be -p0 if the patch was created without an extra prefix in the diff path) patch -p1 < /tmp/yourdownloadedpatch.diff # build binaries only without signing debuild -b -uc # packages are produced in parent directory cd ../ # install all nm packages sudo dpkg -i network-manager*.deb libnm-*.deb }}} Note that you didnt change the version of the package. so to get back to the "official" current version you need to do something like: {{{ sudo apt-get install --reinstall network-manager libnm-util2 libnm-glib4 }}} if you installed other packages from the build, do the same for them. ---- CategoryBugSquad |
If you find a netplan bug, please report it here: https://bugs.launchpad.net/netplan/+filebug |
Available languages: Italiano,
This page is part of the debugging series — pages with debugging details for a variety of Ubuntu packages. |
Bug Summary
If a network-manager bug report is about not being able to connect the title or summary should be in the format:
"[CHIPSET] cannot connect to (ENCRYPT_METHOD)"
where the CHIPSET is the wireless driver used and ENCRYPT_METHOD is the encryption method used by your wireless network.
Understanding your bug and getting more information
There is a lot of debugging information available on the GNOME Live wiki: NetworkManager/Debugging.
You can also take a look at Darren Albers' FAQ.
There is additional information on DebuggingNetworkManager/ReasonCodes for disconnection and network changes available.
Getting debug logs
You can then follow developers' intructions on a bug report for the exact command line to use; or run it directly as such.
Getting NetworkManager debug logs
By default, the NetworkManager log level is set to info. You can use nmcli to modify the logging level:
$ sudo nmcli general logging level DEBUG domains ALL
You do not need to restart NetworkManager to begin seeing debug messages logged to journalctl. You can watch the NetworkManager logs:
$ sudo journalctl -f -u NetworkManager
Getting ModemManager debug logs
Manually run with debug enabled:
$ sudo /usr/sbin/ModemManager --debug $ sudo /usr/sbin/NetworkManager --debug --log-level=DEBUG
See also DebuggingModemmanager.
You do not need to restart ModemManager to begin seeing debug messages logged to journalctl. You can watch the ModemManager logs:
$ sudo journalctl -f -u ModemManager
Getting wpasupplicant debug logs
Change the log level:
$ sudo wpa_cli log_level debug
You do not need to restart wpa_supplicant to begin seeing debug messages logged to journalctl. You can watch the wpa_supplicant logs:
$ sudo journalctl -f -u wpa_supplicant
Getting a capture of syslog
Mixing and mashing the above is perfectly acceptable as well if you want to see how NetworkManager and other parts of the stack interact together.
In order to understand whats going on and track down issues, its good to have a full log. To do so, capture the complete test case and submit the whole file (don't cut out what you think is important). Please add markers in the log file so the bug triager can easily see what actions the user takes at what point of time (this isn't essential, but helps a lot).
To capture the syslog, do:
$ tail -n0 -f /var/log/syslog > /tmp/syslog
and to stop capturing do Ctrl-C (you will have to type your other commands in an other window or tab)
Adding markers is just like adding new lines with an editor that show the triager what happened at what point of time. You can also do this on the fly as you test with the command logger "[ clicked on wireless network 'ubuntu']" .
Example marker:
Sep 6 08:12:30 ... [ clicked on wireless network 'ubuntu'] Sep 6 08:12:31 ... ...
Handling 3G / modem issues
Here are a few extra things that are very helpful to add in case of issues with 3G.
The output of udevadm for tty devices, and output of lsusb:
$ udevadm info --query=all --path=/sys/class/tty/... --attribute-walk
$ lsusb
Captive portal
You can check the status from the cli using:
$ nmcli networking connectivity check
Since 1.38, you can set Environment=NM_LOG_CONCHECK=1 in NetworkManager.service and restart the service to get additional debug logging about connectivity checking.
A Testcase
A good testcase is a step by step instruction to reproduce your bug starting with driver unloaded and NetworkManager stopped.
First, stop NetworkManager and unload your driver:
$ sudo systemctl stop NetworkManager $ sudo modprobe -r DRIVER
Next, load the driver and start NetworkManager:
$ sudo modprobe DRIVER $ sudo systemctl start NetworkManager
Netplan
A configuration abstraction mechanism has added to NetworkManager as of Ubuntu 23.10, called Netplan. There is a whole library of netplan documentation available. However here is a very basic amount of netplan debugging.
Netplan uses the files in /etc/netplan/*.yaml to generate the running profiles you find at /run/NetworkManager/system-connections/*.nmconnection. If you suspect a feature defined in your /etc/netplan/*.yaml files is not being used, it is a good idea to check that the content matches up.
If you find a netplan bug, please report it here: https://bugs.launchpad.net/netplan/+filebug
DebuggingNetworkManager (last edited 2023-06-27 20:21:45 by hellsworth)