[Eisfair] Nach Updates: kernel: BUG: unable to handle kernel paging request at b7b45866
Stefan Heidrich
stefan-in-news at web.de
Di Dez 18 13:40:08 CET 2018
Hallo,
ich habe in der Firma einen Webserver als VM auf einem ESXi 5.5 laufen
(zugewiesene Hardware: 1 CPU, 4 GB RAM, 120 GB HDD an BusLigic Parallel,
1 Netzwerkkarte VMXNET3), der
1.) unseren Ersatzteilkatalog hostet und
2.) ein Perscript für den Dateiaustausch zu Kunden und Partner bietet.
Das Dateiaustausch-Script ist hier beschrieben:
http://www.schaarwaechter.de/dateienaustausch/
Soweit so gut; das läuft seit langer Zeit tadellos.
Gestern habe ich Updates installiert und zwar folgende:
1. n S: 2.0.22 Apache2 Modul: Mod PHP5
2. n S: 2.0.32 The Apache2 Webserver
3. n S: 2.8.11 eisfair base runtime system update to 2.8.11
4. n S: 2.8.4 cURL - Tool for Transferring Data from URLs
5. n S: 2.8.1 Database-common - Menuintegration for databas
6. n S: 3.22.0 eisfair Kernel (Linux 3.16.60-eisfair-1 SMP)
7. n S: 3.8.0 Ghostscript
8. n S: 2.4.3 Library: libaprutil1
9. n S: 2.8.4 Library: C-URL - 4 - tool f. transfer files
10. n S: 2.8.1 Berkeley DB - 5.3 - database routines
11. n S: 2.8.2 Library: GCrypt - 20 - Crypto library
12. n S: 2.8.3 Library: GnuTLS - 30 - GNU Transport Layer Se
13. n S: 2.8.1 Library: qt4-x11 - Qt 4 GUI related libraries
14. n S: 2.8.1 Library: qt4 - C++ Program Library, Core Comp
15. n S: 2.8.4 Library: sqlite3 - 0 - Database Engine
16. n S: 2.8.4 Library: ssh - 4 - A mulitplatform C library
17. n S: 2.8.5 Library: OpenSSL - 1.1
18. n S: 2.8.2 Library: OpenSSL - 1.0.0
19. n S: 1.4.2 NTP Server
20. n S: 2.8.5 OpenSSL - Secure Sockets and Transport Layer
21. n S: 2.8.2 Perl - Interpreter - base
22. n S: 2.8.2 Perl - libwww-perl
23. n S: 2.8.2 Perl - Interpreter
24. n S: 2.0.21 PHP5: CGI - CLI - PEAR - PECL
25. n S: 2.8.4 SQLite3 - Database Engine
Danach kommt es recht häufig vor (ca. 8x), dass der Server einfach
stehen bleibt. Die Meldung auf der Console sieht so aus:
http://www.ispmgh.net/webserver.png.
Beispielhaft der Hänger von 06:40 Uhr; da waren die Logfiles noch recht
überschaubar. In den Logfiles findet man sowas:
/var/log/messages
Dec 18 06:00:14 web-css1 kernel: BUG: unable to handle kernel paging
request at b7b45866
Dec 18 06:00:15 web-css1 kernel: IP:
[vma_interval_tree_insert_after+0x29/0x65]
vma_interval_tree_insert_after+0x29/0x65
Dec 18 06:00:15 web-css1 kernel: *pde = 00000000
Dec 18 06:00:15 web-css1 kernel: Oops: 0000 [#1] SMP
Dec 18 06:00:15 web-css1 kernel: Modules linked in: ipv6 isofs ide_tape
ppdev snd_pcm snd_timer snd soundcore coretemp crc32c_intel pcspkr
vmxnet3 ide_cd_mod cdrom parport_pc parport i2c_piix4 battery i2c_core
ata_generic ac evdev pata_acpi ata_piix libata BusLogic sd_mod scsi_mod
hid_cherry hid_generic usbhid hid xhci_hcd ohci_pci ohci_hcd uhci_hcd
ehci_pci ehci_hcd usbcore usb_common psmouse
Dec 18 06:00:15 web-css1 kernel: CPU: 0 PID: 5309 Comm: httpd Not
tainted 3.16.60-eisfair-1-SMP #1
Dec 18 06:00:15 web-css1 kernel: Hardware name: VMware, Inc. VMware
Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
Dec 18 06:00:15 web-css1 kernel: task: f48f51c0 ti: f500a000 task.ti:
f500a000
Dec 18 06:00:15 web-css1 kernel: EIP:
0060:[vma_interval_tree_insert_after+0x29/0x65] EFLAGS: 00010286 CPU: 0
Dec 18 06:00:15 web-css1 kernel: EIP is at
vma_interval_tree_insert_after+0x29/0x65
Dec 18 06:00:15 web-css1 kernel: EAX: 00000002 EBX: b7b4582e ECX:
b7b4585a EDX: f576bf50
Dec 18 06:00:15 web-css1 kernel: ESI: f5a9c108 EDI: f576bf54 EBP:
f500bf04 ESP: f500befc
Dec 18 06:00:15 web-css1 kernel: DS: 007b ES: 007b FS: 00d8 GS: 00e0
SS: 0068
Dec 18 06:00:15 web-css1 kernel: CR0: 80050033 CR2: b7b45866 CR3:
34e44000 CR4: 001407d0
Dec 18 06:00:15 web-css1 kernel: Stack:
Dec 18 06:00:15 web-css1 kernel: f48bf540 f576bf38 f500bf60 c103080b
00000000 00000000 00000000 f5a9c0c0
Dec 18 06:00:15 web-css1 kernel: f5a9c0c4 f5a9c0b8 f4b51abc f595dabc
f48bf730 f5a9c0b0 00000001 00000000
Und in /var/www/logs/access.log
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:00 +0100] "GET
/template/action.php?func=css&css=style,main,area&cache=2&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 5084
"http://etk.palux.de/template/action.php?func=printPage&page=HOME&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
529 5449
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:00 +0100] "GET
/template/action.php?func=js&js=page&cache=2&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 1987
"http://etk.palux.de/template/action.php?func=printPage&page=HOME&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
536 2346
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/action.php?func=get_constant&name=INFOAREA_NOINFO&cache=2-0&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 50 "http://etk.palux.de/template/index.php" "Mozilla/5.0
(Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 468 409
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 2601 "http://etk.palux.de/template/index.php" "Mozilla/5.0
(Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 494 2980
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/home.png HTTP/1.1" 200 953
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
521 1226
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/zoomIn.png HTTP/1.1" 200 3481
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
523 3755
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/zoomOut.png HTTP/1.1" 200 3442
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
524 3716
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/pan.png HTTP/1.1" 200 699
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
520 972
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/action.php?func=css&css=print&cache=2&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 431
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
535 795
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/action.php?func=js&js=assembly,page,scTbl&cache=2&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 10215
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
567 10593
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/action.php?func=js&js=2d,svg,player&cache=2&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 11228
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
561 11606
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/zoom.png HTTP/1.1" 200 3483
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
521 3757
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/focusPart.png HTTP/1.1" 200 491
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
526 764
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/hlAll.png HTTP/1.1" 200 787
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
522 1060
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/drawing.png HTTP/1.1" 200 486
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
524 759
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/tbl.png HTTP/1.1" 200 581
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
520 854
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/tblEdit.png HTTP/1.1" 200 732
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
524 1005
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/tblFilter.png HTTP/1.1" 200 694
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
526 967
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/fullScreen.png HTTP/1.1" 200 3565
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
527 3839
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/txt.png HTTP/1.1" 200 729
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
520 1002
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/pdf.png HTTP/1.1" 200 665
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
520 938
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/cover.png HTTP/1.1" 200 183
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
514 455
xxx.xxx.xxx.xxx - - [18/Dec/2018:06:00:04 +0100] "GET
/media/ek_blco_vorderseiten_alle.svgz HTTP/1.1" 401 447
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
465 712
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/toolbar/print.png HTTP/1.1" 200 715
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
522 988
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/sort_up.gif HTTP/1.1" 200 64
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
516 335
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/template/images/sort_disabled.gif HTTP/1.1" 200 62
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
522 333
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:04 +0100] "GET
/media/ek_blco_vorderseiten_alle.svgz HTTP/1.1" 200 59570
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
512 59874
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:05 +0100] "POST
/template/action.php?func=tblEdit_attr&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 20 "http://etk.palux.de/template/index.php" "Mozilla/5.0
(Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 572 397
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:13 +0100] "GET
/template/action.php?func=printAssembly&id=2127&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 2426
"http://etk.palux.de/template/action.php?func=printAssembly&id=2097&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
573 2805
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:13 +0100] "GET
/template/images/breadcrumb_hide.gif HTTP/1.1" 200 57
"http://etk.palux.de/template/action.php?func=printAssembly&id=2127&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
524 328
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:13 +0100] "GET
/media/ek_blco_grillpl80_01vorders.svgz HTTP/1.1" 200 6503
"http://etk.palux.de/template/action.php?func=printAssembly&id=2127&highlite=null&tok=9dd127da572b86e8a155d5a5a63415a1"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
514 6806
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:13 +0100] "GET
/template/images/nav/open.png HTTP/1.1" 200 3047
"http://etk.palux.de/template/index.php" "Mozilla/5.0 (Windows NT 6.1;
WOW64; Trident/7.0; rv:11.0) like Gecko" 438 3321
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:13 +0100] "GET
/template/images/nav/cat_open.png HTTP/1.1" 200 511
"http://etk.palux.de/template/index.php" "Mozilla/5.0 (Windows NT 6.1;
WOW64; Trident/7.0; rv:11.0) like Gecko" 442 784
xxx.xxx.xxx.xxx - at_stiasny [18/Dec/2018:06:00:13 +0100] "GET
/template/action.php?func=printNavLevel&id=2097,&cache=1-1-3&tok=9dd127da572b86e8a155d5a5a63415a1
HTTP/1.1" 200 754 "http://etk.palux.de/template/index.php" "Mozilla/5.0
(Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" 459 1113
Ich habe vor den Updates (wie immer) einen Snapshot des Eisfairs gemacht
und habe diesen Snapshot nach der Sicherung der Logfiles wieder
hergestellt. Damit ist der Fehler / die Abstütze wieder verschwunden.
Hat jemand eine Idee wo das kranken könnte? Kernel, Apache2, PHP5, Perl?
Oder wie ich den Fehler eingrenzen könnte?
Viele Grüße
Stefan
Mehr Informationen über die Mailingliste Eisfair