Web lists-archives.com

Help interpreting syslog and konsole messages?




For the last several days (maybe even a week or longer) I've occasionally 
beeng getting messages like the following on all of my open konsole (all tabs, 
all instances) (both root and user).

I also looked into syslog at 18:18:51, and have quoted that further below.

I see the mention of firefox on one line--is this an indication of a firefox 
problem or crash?  (I get firefox crashes, but ususally not one or two a day 
(more like 2 or 3 a month).

What is a CPU stall?

Can anyone give me some insight into what is going on?

Captured from a konsole:

<quote>
rhk@s19:/rhk/bin/backups_rotate$ 
Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564150] Stack:

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564160] Call Trace:

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564177] Code: ee 48 c7 c7 80 a4 4c 81 31 c0 e8 6c 33 32 00 48 
83 c4 18 89 d8 5b 5d 41 5c 41 5d c3 90 90 90 9c 58 c3 57 9d c3 fa c3 fb c3 fb 
f4 <c3> f4 c3 66 66 66 90 66 66 90 c3 66 66 66 90 66 66 90 c3 0f 06 

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564472] Stack:

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564507] Call Trace:

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564512]  <IRQ> 

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564642]  <EOI> 

Message from syslogd@s19 at Feb 10 01:10:07 ...
 kernel:[1692814.564692] Code: 66 00 31 c0 f7 c2 00 00 20 00 74 18 48 8d 74 24 
0c bf 1b 00 00 00 e8 ab fb ff ff f6 c4 04 0f 95 c0 0f b6 c0 48 83 c4 10 c3 90 57 
<9d> 66 66 90 66 90 c3 89 ff 89 b7 00 b0 5f ff c3 89 ff 8b 87 00 

<ignore this>
rhk@s19:/rhk/bin/backups_rotate$ ping -6 security.debian.org
ping: invalid option -- '6'
Usage: ping [-LRUbdfnqrvVaAD] [-c count] [-i interval] [-w deadline]
            [-p pattern] [-s packetsize] [-t ttl] [-I interface]
            [-M pmtudisc-hint] [-m mark] [-S sndbuf]
            [-T tstamp-options] [-Q tos] [hop1 ...] destination
rhk@s19:/rhk/bin/backups_rotate$ 
</ignore this>

Message from syslogd@s19 at Feb 10 18:15:51 ...
 kernel:[1754358.244408] Stack:

Message from syslogd@s19 at Feb 10 18:15:51 ...
 kernel:[1754358.244443] Call Trace:

Message from syslogd@s19 at Feb 10 18:15:51 ...
 kernel:[1754358.244448]  <IRQ> 

Message from syslogd@s19 at Feb 10 18:15:51 ...
 kernel:[1754358.244570]  <EOI> 

Message from syslogd@s19 at Feb 10 18:15:51 ...
 kernel:[1754358.244620] Code: 66 00 31 c0 f7 c2 00 00 20 00 74 18 48 8d 74 24 
0c bf 1b 00 00 00 e8 ab fb ff ff f6 c4 04 0f 95 c0 0f b6 c0 48 83 c4 10 c3 90 57 
<9d> 66 66 90 66 90 c3 89 ff 89 b7 00 b0 5f ff c3 89 ff 8b 87 00 
</quote>

From syslog:

<quote>
Feb 10 07:35:12 s19 anacron[19011]: Job `cron.daily' terminated
Feb 10 07:35:12 s19 anacron[19011]: Normal exit (1 job run)
... <ignore the clearing of .xsession-errors files>
Feb 10 18:15:01 s19 /USR/SBIN/CRON[25926]: (root) CMD (echo "Cleared on 
$(date) by $USER cron" > /home/rhk/.xsession-errors)
Feb 10 18:15:01 s19 /USR/SBIN/CRON[25927]: (root) CMD (echo "Cleared on 
$(date) by $USER cron" > /root/.xsession-errors)

Feb 10 18:15:51 s19 kernel: [1754358.244038] INFO: rcu_bh detected stall on 
CPU 1 (t=0 jiffies)
Feb 10 18:15:51 s19 kernel: [1754358.244049] sending NMI to all CPUs:
Feb 10 18:15:51 s19 kernel: [1754358.244059] NMI backtrace for cpu 0
Feb 10 18:15:51 s19 kernel: [1754358.244062] CPU 0 
Feb 10 18:15:51 s19 kernel: [1754358.244063] Modules linked in: snd_seq_dummy 
snd_hrtimer usblp sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt 
dm_mod parport_pc ppdev lp parport nfsd nfs nfs_acl auth_rpcgss fscache lockd 
sunrpc nls_utf8 nls_cp437 vfat fat ext2 loop snd_hda_codec_hdmi sp5100_tco 
snd_hda_codec_via snd_usb_audio snd_usbmidi_lib snd_seq_midi 
snd_seq_midi_event radeon i2c_piix4 powernow_k8 edac_mce_amd mperf k10temp 
psmouse evdev snd_rawmidi serio_raw edac_core snd_hda_intel snd_hda_codec 
snd_hwdep snd_seq pcspkr snd_seq_device snd_pcm snd_page_alloc snd_timer ttm 
snd drm_kms_helper drm power_supply i2c_algo_bit i2c_core soundcore shpchp wmi 
button processor ext4 crc16 jbd2 mbcache sr_mod sg cdrom sd_mod crc_t10dif 
ata_generic thermal thermal_sys ohci_hcd atl1c pata_atiixp ahci libahci 
ehci_hcd usbcore libata usb_common scsi_mod [last unloaded: scsi_wait_scan]
Feb 10 18:15:51 s19 kernel: [1754358.244108] 
Feb 10 18:15:51 s19 kernel: [1754358.244111] Pid: 6701, comm: firefox-esr Not 
tainted 3.2.0-5-amd64 #1 Debian 3.2.96-3 ECS A880GM-M7/A880GM-M7
Feb 10 18:15:51 s19 kernel: [1754358.244114] RIP: 0033:[<00007f887d5ba579>]  
[<00007f887d5ba579>] 0x7f887d5ba578
Feb 10 18:15:51 s19 kernel: [1754358.244120] RSP: 002b:00007ffcc4565d80  
EFLAGS: 00000246
Feb 10 18:15:51 s19 kernel: [1754358.244122] RAX: 00007f86a6a15128 RBX: 
0000000000000000 RCX: 00007f86a6a62000
Feb 10 18:15:51 s19 kernel: [1754358.244124] RDX: 00007f86a6a15128 RSI: 
00007f86a7200000 RDI: 00007ffcc4565d90
Feb 10 18:15:51 s19 kernel: [1754358.244125] RBP: 00007ffcc4565d80 R08: 
00007f8889800048 R09: 0000000000001a2d
Feb 10 18:15:51 s19 kernel: [1754358.244127] R10: 00007f8646e36160 R11: 
0000000000000283 R12: 00007ffcc4565d88
Feb 10 18:15:51 s19 kernel: [1754358.244128] R13: 00007ffcc4565d90 R14: 
00007f869feb86e8 R15: 00007f869feb86f0
Feb 10 18:15:51 s19 kernel: [1754358.244131] FS:  00007f8889bef720(0000) 
GS:ffff88041fc00000(0000) knlGS:0000000000000000
Feb 10 18:15:51 s19 kernel: [1754358.244133] CS:  0010 DS: 0000 ES: 0000 CR0: 
0000000080050033
Feb 10 18:15:51 s19 kernel: [1754358.244134] CR2: 00007f868044e010 CR3: 
000000040d4d8000 CR4: 00000000000006f0
Feb 10 18:15:51 s19 kernel: [1754358.244136] DR0: 0000000000000000 DR1: 
0000000000000000 DR2: 0000000000000000
Feb 10 18:15:51 s19 kernel: [1754358.244138] DR3: 0000000000000000 DR6: 
00000000ffff0ff0 DR7: 0000000000000400
Feb 10 18:15:51 s19 kernel: [1754358.244140] Process firefox-esr (pid: 6701, 
threadinfo ffff880306c02000, task ffff88031eaf4740)
Feb 10 18:15:51 s19 kernel: [1754358.244141] 
Feb 10 18:15:51 s19 kernel: [1754358.244142] Call Trace:
Feb 10 18:15:51 s19 kernel: [1754358.244145] NMI backtrace for cpu 1
Feb 10 18:15:51 s19 kernel: [1754358.244150] CPU 1 
Feb 10 18:15:51 s19 kernel: [1754358.244154] Modules linked in: snd_seq_dummy 
snd_hrtimer usblp sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt 
dm_mod parport_pc ppdev lp parport nfsd nfs nfs_acl auth_rpcgss fscache lockd 
sunrpc nls_utf8 nls_cp437 vfat fat ext2 loop snd_hda_codec_hdmi sp5100_tco 
snd_hda_codec_via snd_usb_audio snd_usbmidi_lib snd_seq_midi 
snd_seq_midi_event radeon i2c_piix4 powernow_k8 edac_mce_amd mperf k10temp 
psmouse evdev snd_rawmidi serio_raw edac_core snd_hda_intel snd_hda_codec 
snd_hwdep snd_seq pcspkr snd_seq_device snd_pcm snd_page_alloc snd_timer ttm 
snd drm_kms_helper drm power_supply i2c_algo_bit i2c_core soundcore shpchp wmi 
button processor ext4 crc16 jbd2 mbcache sr_mod sg cdrom sd_mod crc_t10dif 
ata_generic thermal thermal_sys ohci_hcd atl1c pata_atiixp ahci libahci 
ehci_hcd usbcore libata usb_common scsi_mod [last unloaded: scsi_wait_scan]
Feb 10 18:15:51 s19 kernel: [1754358.244294] 
Feb 10 18:15:51 s19 kernel: [1754358.244302] Pid: 0, comm: swapper/1 Not 
tainted 3.2.0-5-amd64 #1 Debian 3.2.96-3 ECS A880GM-M7/A880GM-M7
Feb 10 18:15:51 s19 kernel: [1754358.244313] RIP: 0010:[<ffffffff8102847d>]  
[<ffffffff8102847d>] arch_local_irq_restore+0x1/0x8
Feb 10 18:15:51 s19 kernel: [1754358.244331] RSP: 0018:ffff88041fc43e18  EFLAGS: 
00000046
Feb 10 18:15:51 s19 kernel: [1754358.244337] RAX: 0000000000000000 RBX: 
0000000000002710 RCX: 000000000000dbc1
Feb 10 18:15:51 s19 kernel: [1754358.244344] RDX: 0000000000000000 RSI: 
0000000000000c00 RDI: 0000000000000046
Feb 10 18:15:51 s19 kernel: [1754358.244350] RBP: ffffffff81623100 R08: 
0000000000000002 R09: 0000000000000000
Feb 10 18:15:51 s19 kernel: [1754358.244356] R10: 000000002541b503 R11: 
000000002541b503 R12: ffffffff81623200
Feb 10 18:15:51 s19 kernel: [1754358.244362] R13: ffff88041fc43f50 R14: 
ffff88041fc50678 R15: ffffffff8106e211
Feb 10 18:15:51 s19 kernel: [1754358.244370] FS:  00007f8873efc700(0000) 
GS:ffff88041fc40000(0000) knlGS:0000000000000000
Feb 10 18:15:51 s19 kernel: [1754358.244377] CS:  0010 DS: 0000 ES: 0000 CR0: 
000000008005003b
Feb 10 18:15:51 s19 kernel: [1754358.244383] CR2: 00007f86c89e9000 CR3: 
000000040d4d8000 CR4: 00000000000006f0
Feb 10 18:15:51 s19 kernel: [1754358.244389] DR0: 0000000000000000 DR1: 
0000000000000000 DR2: 0000000000000000
Feb 10 18:15:51 s19 kernel: [1754358.244396] DR3: 0000000000000000 DR6: 
00000000ffff0ff0 DR7: 0000000000000400
Feb 10 18:15:51 s19 kernel: [1754358.244403] Process swapper/1 (pid: 0, 
threadinfo ffff88040f4fa000, task ffff88040f4df180)
Feb 10 18:15:51 s19 kernel: [1754358.244408] Stack:
Feb 10 18:15:51 s19 kernel: [1754358.244412]  0000000000000046 ffffffff81025602 
ffff88041fc508a0 ffffffff8109833c
Feb 10 18:15:51 s19 kernel: [1754358.244424]  ffff88040ad6dd58 0000000000000092 
ffffffff81011a01 0000000000000001
Feb 10 18:15:51 s19 kernel: [1754358.244434]  0000000000000000 
0000000000000001 ffff88041fc43f50 ffff88041fc50678
Feb 10 18:15:51 s19 kernel: [1754358.244443] Call Trace:
Feb 10 18:15:51 s19 kernel: [1754358.244448]  <IRQ> 
Feb 10 18:15:51 s19 kernel: [1754358.244460]  [<ffffffff81025602>] ? 
arch_trigger_all_cpu_backtrace+0x4d/0x7b
Feb 10 18:15:51 s19 kernel: [1754358.244471]  [<ffffffff8109833c>] ? 
__rcu_pending+0x82/0x358
Feb 10 18:15:51 s19 kernel: [1754358.244480]  [<ffffffff81011a01>] ? 
arch_local_irq_save+0x5/0x13
Feb 10 18:15:51 s19 kernel: [1754358.244491]  [<ffffffff8106e211>] ? 
tick_nohz_handler+0xcd/0xcd
Feb 10 18:15:51 s19 kernel: [1754358.244499]  [<ffffffff8109895e>] ? 
rcu_check_callbacks+0xaf/0xcc
Feb 10 18:15:51 s19 kernel: [1754358.244508]  [<ffffffff81054874>] ? 
update_process_times+0x31/0x63
Feb 10 18:15:51 s19 kernel: [1754358.244517]  [<ffffffff8106e278>] ? 
tick_sched_timer+0x67/0x8d
Feb 10 18:15:51 s19 kernel: [1754358.244526]  [<ffffffff810643e8>] ? 
__run_hrtimer+0xc4/0x161
Feb 10 18:15:51 s19 kernel: [1754358.244535]  [<ffffffff81064b03>] ? 
hrtimer_interrupt+0xd7/0x1b1
Feb 10 18:15:51 s19 kernel: [1754358.244544]  [<ffffffff81014124>] ? 
sched_clock+0x5/0x8
Feb 10 18:15:51 s19 kernel: [1754358.244554]  [<ffffffff81024c0b>] ? 
smp_apic_timer_interrupt+0x6f/0x82
Feb 10 18:15:51 s19 kernel: [1754358.244565]  [<ffffffff8135bd22>] ? 
apic_timer_interrupt+0x82/0x90
Feb 10 18:15:51 s19 kernel: [1754358.244570]  <EOI> 
Feb 10 18:15:51 s19 kernel: [1754358.244577]  [<ffffffff81098a19>] ? 
rcu_needs_cpu+0x50/0x1bb
Feb 10 18:15:51 s19 kernel: [1754358.244586]  [<ffffffff8102c370>] ? 
native_safe_halt+0x2/0x3
Feb 10 18:15:51 s19 kernel: [1754358.244595]  [<ffffffff81014c40>] ? 
default_idle+0x47/0x7f
Feb 10 18:15:51 s19 kernel: [1754358.244604]  [<ffffffff8100d24c>] ? 
cpu_idle+0xaf/0xf2
Feb 10 18:15:51 s19 kernel: [1754358.244614]  [<ffffffff8134805b>] ? 
start_secondary+0x1da/0x1e0
Feb 10 18:15:51 s19 kernel: [1754358.244620] Code: 66 00 31 c0 f7 c2 00 00 20 
00 74 18 48 8d 74 24 0c bf 1b 00 00 00 e8 ab fb ff ff f6 c4 04 0f 95 c0 0f b6 c0 
48 83 c4 10 c3 90 57 <9d> 66 66 90 66 90 c3 89 ff 89 b7 00 b0 5f ff c3 89 ff 8b 
87 00 
Feb 10 18:15:51 s19 kernel: [1754358.244695] Call Trace:
Feb 10 18:15:51 s19 kernel: [1754358.244699]  <IRQ>  [<ffffffff81025602>] ? 
arch_trigger_all_cpu_backtrace+0x4d/0x7b
Feb 10 18:15:51 s19 kernel: [1754358.244714]  [<ffffffff8109833c>] ? 
__rcu_pending+0x82/0x358
Feb 10 18:15:51 s19 kernel: [1754358.244722]  [<ffffffff81011a01>] ? 
arch_local_irq_save+0x5/0x13
Feb 10 18:15:51 s19 kernel: [1754358.244731]  [<ffffffff8106e211>] ? 
tick_nohz_handler+0xcd/0xcd
Feb 10 18:15:51 s19 kernel: [1754358.244739]  [<ffffffff8109895e>] ? 
rcu_check_callbacks+0xaf/0xcc
Feb 10 18:15:51 s19 kernel: [1754358.244747]  [<ffffffff81054874>] ? 
update_process_times+0x31/0x63
Feb 10 18:15:51 s19 kernel: [1754358.244756]  [<ffffffff8106e278>] ? 
tick_sched_timer+0x67/0x8d
Feb 10 18:15:51 s19 kernel: [1754358.244763]  [<ffffffff810643e8>] ? 
__run_hrtimer+0xc4/0x161
Feb 10 18:15:51 s19 kernel: [1754358.244772]  [<ffffffff81064b03>] ? 
hrtimer_interrupt+0xd7/0x1b1
Feb 10 18:15:51 s19 kernel: [1754358.244780]  [<ffffffff81014124>] ? 
sched_clock+0x5/0x8
Feb 10 18:15:51 s19 kernel: [1754358.244789]  [<ffffffff81024c0b>] ? 
smp_apic_timer_interrupt+0x6f/0x82
Feb 10 18:15:51 s19 kernel: [1754358.244798]  [<ffffffff8135bd22>] ? 
apic_timer_interrupt+0x82/0x90
Feb 10 18:15:51 s19 kernel: [1754358.244803]  <EOI>  [<ffffffff81098a19>] ? 
rcu_needs_cpu+0x50/0x1bb
Feb 10 18:15:51 s19 kernel: [1754358.244816]  [<ffffffff8102c370>] ? 
native_safe_halt+0x2/0x3
Feb 10 18:15:51 s19 kernel: [1754358.244824]  [<ffffffff81014c40>] ? 
default_idle+0x47/0x7f
Feb 10 18:15:51 s19 kernel: [1754358.244832]  [<ffffffff8100d24c>] ? 
cpu_idle+0xaf/0xf2
Feb 10 18:15:51 s19 kernel: [1754358.244842]  [<ffffffff8134805b>] ? 
start_secondary+0x1da/0x1e0

Feb 10 18:16:01 s19 /USR/SBIN/CRON[25958]: (root) CMD (echo "Cleared on 
$(date) by $USER cron" > /root/.xsession-errors)
Feb 10 18:16:01 s19 /USR/SBIN/CRON[25959]: (root) CMD (echo "Cleared on 
$(date) by $USER cron" > /home/rhk/.xsession-errors)
</quote>