DebuggingNetworkManager
|
Size: 4559
Comment:
|
Size: 4564
Comment: Debug hints for captive portal issues
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| == A Testcase == | Available languages: [[https://wiki.ubuntu.com/DebuggingNetworkManager_it| Italiano]], <<Include(Debugging/Header)>> ||<tablestyle="float:right; font-size: 0.9em; width:30%; background:#F1F1ED; background-image: url('https://librarian.launchpad.net/1812570/bugsquad.png'); background-repeat: no-repeat; background-position: 98% 0.5ex; margin: 0 0 1em 1em; padding: 0.5em;"><<TableOfContents>>|| |
| Line 3: | Line 5: |
| A good testcase is a step by step instruction to reproduce your bug starting with driver unloaded and NetworkManager stopped. | = Bug Summary = |
| Line 5: | Line 7: |
| Kill NetworkManager | If a network-manager bug report is about not being able to connect the title or summary should be in the format: "[CHIPSET] cannot connect to (ENCRYPT_METHOD)" where the CHIPSET is the wireless driver used and ENCRYPT_METHOD is the encryption method used by your wireless network. = Understanding your bug and getting more information = * There is a lot of debugging information available on the GNOME Live wiki: [[https://wiki.gnome.org/Projects/NetworkManager/Debugging|NetworkManager/Debugging]]. * You can also take a look at [[http://live.gnome.org/DarrenAlbers/NetworkManagerFAQ|Darren Albers' FAQ]]. * There is additional information on DebuggingNetworkManager/ReasonCodes for disconnection and network changes available. * The similar reason/status code data for wpasupplicant is available here: [[http://hostap.epitest.fi/gitweb/gitweb.cgi?p=hostap.git;a=blob;f=src/common/ieee802_11_defs.h|ieee802_11_defs.h]] = Getting debug logs = You can then follow developers' intructions on a bug report for the exact command line to use; or run it directly as such: == Getting NetworkManager debug logs == |
| Line 7: | Line 28: |
| sudo killall NetworkManager | sudo /usr/lib/NetworkManager/debug-helper.py --nm debug |
| Line 10: | Line 31: |
| To unload your driver {{{ modprobe -r DRIVER }}}. | Then get the logs which will be written to /var/log/syslog. To disable it, pass '''info''' instead of '''debug''' in the above command, or reboot. |
| Line 12: | Line 33: |
| Then load the driver and start NetworkManager: | == Getting ModemManager debug logs == |
| Line 14: | Line 36: |
| sudo NetworkManager | sudo /usr/lib/NetworkManager/debug-helper.py --mm debug |
| Line 17: | Line 39: |
| === Capture Log === | Then get the logs which will be written to /var/log/syslog. To disable it, pass '''info''' instead of '''debug''' in the above command, or reboot. See also [[DebuggingModemmanager]]. == Getting wpasupplicant debug logs == {{{ sudo /usr/lib/NetworkManager/debug-helper.py --wpa msgdump }}} Then get the logs which will be written to /var/log/syslog. To disable it, pass '''info''' instead of '''debug''' in the above command, or reboot. == Getting a capture of syslog == Mixing and mashing the above is perfectly acceptable as well if you want to see how NetworkManager and other parts of the stack interact together. |
| Line 26: | Line 62: |
| Adding markers is just like adding new lines with an editor that show the triager what happened at what point of time. | Adding markers is just like adding new lines with an editor that show the triager what happened at what point of time. You can also do this on the fly as you test with the command {{{logger "[ clicked on wireless network 'ubuntu']" }}}. |
| Line 33: | Line 69: |
| Line 37: | Line 74: |
| == Debugging Crashes == | == Handling 3G / modem issues == |
| Line 39: | Line 76: |
| To install debug symbols, add the following line to your {{{/etc/apt/sources.list}}} | An few extra things that are very helpful to add in case of issues with 3G: The output of udevadm for tty devices, and output of lsusb: |
| Line 42: | Line 81: |
| deb http://ddebs.ubuntu.com/~ubuntu-archive/ddebs/ gutsy main universe | $ udevadm info --query=all --path=/sys/class/tty/... --attribute-walk |
| Line 45: | Line 84: |
| Then install the appropriate dbgsym packages: |
|
| Line 48: | Line 85: |
| sudo apt-get update sudo apt-get install network-manager-dbgsym libnm-util0-dbgsym libnm-glib0-dbgsym libglib2.0-0-dbgsym |
$ lsusb |
| Line 52: | Line 88: |
| Then restart NetworkManager: | == Captive portal == You can check the status from the cli using |
| Line 54: | Line 92: |
| sudo /etc/dbus-1/event.d/25NetworkManager restart | $ nmcli networking connectivity check |
| Line 57: | Line 95: |
| Attach the debugger to the pid of NetworkManager | The server should return that value |
| Line 59: | Line 97: |
| sudo gdb /usr/bin/NetworkManager $(pidof NetworkManager) ... (gdb) continue |
$ wget -O - http://connectivity-check.ubuntu.com./ -d 2>&1 | grep X- X-NetworkManager-Status: online |
| Line 64: | Line 101: |
| Once it crashes get a backtrace {{{ (gdb) bt ... (gdb) bt full ... (gdb) thread apply all bt full ... |
Since 1.38, set Environment=NM_LOG_CONCHECK=1 in NetworkManager.service and restart the service to get additional debug logging about connectivity checking. = A Testcase = A good testcase is a step by step instruction to reproduce your bug starting with driver unloaded and NetworkManager stopped. First, stop NetworkManager: {{{ sudo service network-manager stop }}} Then unload your driver:{{{ sudo modprobe -r DRIVER }}} Next, load the driver:{{{ sudo modprobe DRIVER }}} and finally start NetworkManager:{{{ sudo service network-manager start |
| Line 73: | Line 116: |
and attach the backtrace above together with your {{{/var/log/syslog}}} to the bug. == Driver Logs == When a bug appears to be driver related or you are asked by a bug triager to submit a driver enabled log, you need to enable driver logging right before you start to capture your testcase. How to do that depends on the driver you use and whether it has been with compiled with debug support. == Driver Specific Info == === IPW (2100,2200, 3945) === ==== Logging ==== {{{ # either during module load: modprobe ipw{2100,2200,3945} debug=65535 # or when already loaded you can change the debug_level through /sys/bus/.../drivers/ echo 65535 > /sys/bus/pci/drivers/ipw{2100,2200,3945}/debug_level }}} ==== Compiling Module Sources ==== For debugging purpose or to verify a fix, a developer might ask you to build your driver module from source. for ipwXXXX you can do that by: {{{ # install required headers and build tools sudo apt-get install module-assistant sudo module assistant update sudo module assistant prepare # build the driver cd /path/to/ipw-XXXX make IEEE80211_IGNORE_DUPLICATE=y SHELL=/bin/bash # backup your old ipw driver: sudo cp /lib/modules/$(uname -r)/kernel/drivers/net/wireless/ipw3945/ipw3945.ko $HOME # install the new driver cp ipw3945.ko /lib/modules/$(uname -r)/kernel/drivers/net/wireless/ipw3945/ depmod -a # kill regulatory daemon ipw3945d-$(uname -r) --kill #load new module modprobe ipw3945 }}} == Bug Tags == This tags allows to isolate bugs into smaller groups, providing an easiest and faster way to work on specific issues. || Tag || Use case || Link || || `driver-madwifi` || Bugs in which the madwifi driver is in use || [https://launchpad.net/ubuntu/+bugs?field.tag=driver-madwifi link] || || `driver-ipw2100`|| Bugs in which the ipw2100 driver is in use || [https://launchpad.net/ubuntu/+bugs?field.tag=driver-ipw2100 link] || || `driver-ndiswrapper`|| Bugs in which the ndiswrapper driver is in use || [https://launchpad.net/ubuntu/+bugs?field.tag=driver-ndiswrapper link] || || `vpn`|| Bugs related to either openvpn, vpnc or pptp network-manager vpn modules || [https://launchpad.net/ubuntu/+bugs?field.tag=vpn link] || || `encryption-wep`|| Bugs in which WEP encryption is used || [https://launchpad.net/ubuntu/+bugs?field.tag=encryption-wep link] || || `encryption-wpa`|| Bugs in which WPA encryption is used || [https://launchpad.net/ubuntu/+bugs?field.tag=encryption-wpa link] || || `encryption-wpa2`|| Bugs in which WPA2 encryption is used || [https://launchpad.net/ubuntu/+bugs?field.tag=encryption-wpa2 link] || ---- CategoryBugSquad |
Available languages: Italiano,
This page is part of the debugging series — pages with debugging details for a variety of Ubuntu packages. |
Bug Summary
If a network-manager bug report is about not being able to connect the title or summary should be in the format:
"[CHIPSET] cannot connect to (ENCRYPT_METHOD)"
where the CHIPSET is the wireless driver used and ENCRYPT_METHOD is the encryption method used by your wireless network.
Understanding your bug and getting more information
There is a lot of debugging information available on the GNOME Live wiki: NetworkManager/Debugging.
You can also take a look at Darren Albers' FAQ.
There is additional information on DebuggingNetworkManager/ReasonCodes for disconnection and network changes available.
The similar reason/status code data for wpasupplicant is available here: ieee802_11_defs.h
Getting debug logs
You can then follow developers' intructions on a bug report for the exact command line to use; or run it directly as such:
Getting NetworkManager debug logs
sudo /usr/lib/NetworkManager/debug-helper.py --nm debug
Then get the logs which will be written to /var/log/syslog. To disable it, pass info instead of debug in the above command, or reboot.
Getting ModemManager debug logs
sudo /usr/lib/NetworkManager/debug-helper.py --mm debug
Then get the logs which will be written to /var/log/syslog. To disable it, pass info instead of debug in the above command, or reboot.
See also DebuggingModemmanager.
Getting wpasupplicant debug logs
sudo /usr/lib/NetworkManager/debug-helper.py --wpa msgdump
Then get the logs which will be written to /var/log/syslog. To disable it, pass info instead of debug in the above command, or reboot.
Getting a capture of syslog
Mixing and mashing the above is perfectly acceptable as well if you want to see how NetworkManager and other parts of the stack interact together.
In order to understand whats going on and track down issues, its good to have a full log. To do so, capture the complete test case and submit the whole file (don't cut out what you think is important). Please add markers in the log file so the bug triager can easily see what actions the user takes at what point of time (this isn't essential, but helps a lot).
To capture the syslog, do:
tail -n0 -f /var/log/syslog > /tmp/syslog
and to stop capturing do Ctrl-C (you will have to type your other commands in an other window or tab)
Adding markers is just like adding new lines with an editor that show the triager what happened at what point of time. You can also do this on the fly as you test with the command logger "[ clicked on wireless network 'ubuntu']" .
Example marker:
Sep 6 08:12:30 ... [ clicked on wireless network 'ubuntu'] Sep 6 08:12:31 ... ...
Handling 3G / modem issues
An few extra things that are very helpful to add in case of issues with 3G:
The output of udevadm for tty devices, and output of lsusb:
$ udevadm info --query=all --path=/sys/class/tty/... --attribute-walk
$ lsusb
Captive portal
You can check the status from the cli using
$ nmcli networking connectivity check
The server should return that value
$ wget -O - http://connectivity-check.ubuntu.com./ -d 2>&1 | grep X- X-NetworkManager-Status: online
Since 1.38, set Environment=NM_LOG_CONCHECK=1 in NetworkManager.service and restart the service to get additional debug logging about connectivity checking.
A Testcase
A good testcase is a step by step instruction to reproduce your bug starting with driver unloaded and NetworkManager stopped.
First, stop NetworkManager:
sudo service network-manager stop
Then unload your driver:
sudo modprobe -r DRIVER
Next, load the driver:
sudo modprobe DRIVER
and finally start NetworkManager:
sudo service network-manager start
DebuggingNetworkManager (last edited 2023-06-27 20:21:45 by hellsworth)