[Driftstörning] Problem med webhotellet 23-24 februari

[Patrick Forsberg]

(English translation below)
Hej!

Vi har haft problem med webhotellet under gårdagen och idag på morgonen. 
Problemet var att resurserna tog slut och därmed slutade tjänsten att svara
Vi har lokaliserat och åtgärdat problemet.

Detaljerad beskrivning.

Serverkonfigurationen tillåter ett maximalt antal server-processer 
(ServerLimit) och ett maximalt antal anslutna klienter (MaxClients)
Det vi noterade var att servern loggade att gränsen för MaxClients var 
nådd och att det fanns lika många processer som körde.
Innan vi lokaliserat felet var nödåtgärden att starta om webtjänsten 
vilket gav en viss respit tills MaxClients åter var uppnått.
Slutligen lokaliserades felet till att processerna var fast i "sending 
reply", vilket kunde kopplas till att automonteringen av en 
hemsideskatalog inte fungerade som den skulle.
Omstart av automountern har nu löst problemet och allt skall fungera som 
vanligt. Vi kommer att lägga in övervakning för att försöka fånga upp 
problemet om det händer igen.

+++

Hi,

There has been a reccurring problem with the web hotell yesterday and 
this morning. The problem was that it ran out of resources which led to 
the service not responding.
We have identified the root cause of the problem and fixed it.

Detailed description.

The server config allows for a maximum number of processes (ServerLimit) 
and a maximum amount of connected Clients (MaxClients)
What we noted were logs stating that the limit for MaxClients was 
reached and also that there was a corresponding amount of processes 
still lingering.
Before the cause of the problem was diagnosed, the quick fix was to 
restart the web service which bought some time but eventually MaxClients 
was reached again.
Finally we tracked down the problem to processes being stuck in "sending 
reply" which meant that they couldn't finish. The reason they stuck was 
a homepage directory not working as it should. Restarting the 
automounter process solved this problem and things should now be back to 
normal. We will put in monitoring to try and catch this kind of problem 
if it should ever occur again.


Hälsningar,

IT-Avdelningen
Patrick Forsberg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3615 bytes
Desc: S/MIME Cryptographic Signature
Url : http://lists.chalmers.se/pipermail/driftstorning/attachments/20160224/93bc0666/smime.bin